Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #72093 > unrolled thread

Command prompt not shown when running Python script with subprocess on Windows

Back to article view | Back to comp.lang.python

Contents

  Command prompt not shown when running Python script with subprocess on Windows ps16thypresenceisfullnessofjoy@gmail.com - 2014-05-26 17:03 -0700
    Re:Command prompt not shown when running Python script with subprocess on Windows Dave Angel <davea@davea.name> - 2014-05-27 03:20 -0400
    Re: Command prompt not shown when running Python script with subprocess on Windows Stephen Hansen <me+python@ixokai.io> - 2014-05-26 23:38 -0700
      Re: Command prompt not shown when running Python script with subprocess on Windows ps16thypresenceisfullnessofjoy@gmail.com - 2014-05-27 16:01 -0700
        Re: Command prompt not shown when running Python script with subprocess on Windows Tim Golden <mail@timgolden.me.uk> - 2014-05-28 06:08 +0100
          Re: Command prompt not shown when running Python script with subprocess on Windows ps16thypresenceisfullnessofjoy@gmail.com - 2014-05-28 13:22 -0700
        Re: Command prompt not shown when running Python script with subprocess on Windows Tim Golden <mail@timgolden.me.uk> - 2014-05-28 20:46 +0100
          Re: Command prompt not shown when running Python script with subprocess on Windows ps16thypresenceisfullnessofjoy@gmail.com - 2014-05-28 13:46 -0700
            Re: Command prompt not shown when running Python script with subprocess on Windows Tim Golden <mail@timgolden.me.uk> - 2014-05-29 12:41 +0100
              Re: Command prompt not shown when running Python script with subprocess on Windows ps16thypresenceisfullnessofjoy@gmail.com - 2014-05-29 12:21 -0700
                Re: Command prompt not shown when running Python script with subprocess on Windows Tim Golden <mail@timgolden.me.uk> - 2014-06-01 17:31 +0100
                  Re: Command prompt not shown when running Python script with subprocess on Windows ps16thypresenceisfullnessofjoy@gmail.com - 2014-06-04 13:25 -0700

#72093 — Command prompt not shown when running Python script with subprocess on Windows

I have written a Python script with a wxPython GUI that uses subprocess.Popen to open a list of files that the user provides. One of my users would like to be able to run a Python script with my application. The Python script he is trying to run uses the command line and gets keyboard input from the user several times.

The problem is that if the Python script is run on Windows with subprocess.Popen, no command prompt is shown (my GUI application is a .pyw file). The user's script runs silently but then does not quit because it is waiting for input, but there is no way for the input to be given, since there is no command prompt visible.

I think this may be related to the fact that I am calling subprocess.Popen with shell=True. I tried calling it with shell=False (the default), but then I got an error that the file is not a valid Win32 application.

I would appreciate any help with this problem.

-- Timothy

[toc] | [next] | [standalone]

#72102

ps16thypresenceisfullnessofjoy@gmail.com Wrote in message:
> I have written a Python script with a wxPython GUI that uses subprocess.Popen to open a list of files that the user provides. One of my users would like to be able to run a Python script with my application. The Python script he is trying to run uses the command line and gets keyboard input from the user several times.
> 
> The problem is that if the Python script is run on Windows with subprocess.Popen, no command prompt is shown (my GUI application is a .pyw file). The user's script runs silently but then does not quit because it is waiting for input, but there is no way for the input to be given, since there is no command prompt visible.
> 
> I think this may be related to the fact that I am calling subprocess.Popen with shell=True. I tried calling it with shell=False (the default), but then I got an error that the file is not a valid Win32 application.
> 
> I would appreciate any help with this problem.
> 
> -- Timothy
> 

If you want to use shell=False, you need to specify the executable
 correctly.  Since you're on Windows,  the executable is named
 python.exe, not myscript. py

If you still get errors,  you need to get a lot more explicit. 
 Copy/paste, not paraphrase. 

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#72103

[Multipart message — attachments visible in raw view] — view raw

You need to call python.exe path-to-script.py, I think, not just
path-to-script.py. See sys.executable (though that depends on if you're a
frozen app or not).

I can't be sure though because there's no code. Show code when asking
questions, it helps frame the discussion and get a better answer ;)


On Mon, May 26, 2014 at 5:03 PM,
<ps16thypresenceisfullnessofjoy@gmail.com>wrote:

> I have written a Python script with a wxPython GUI that uses
> subprocess.Popen to open a list of files that the user provides. One of my
> users would like to be able to run a Python script with my application. The
> Python script he is trying to run uses the command line and gets keyboard
> input from the user several times.
>
> The problem is that if the Python script is run on Windows with
> subprocess.Popen, no command prompt is shown (my GUI application is a .pyw
> file). The user's script runs silently but then does not quit because it is
> waiting for input, but there is no way for the input to be given, since
> there is no command prompt visible.
>
> I think this may be related to the fact that I am calling subprocess.Popen
> with shell=True. I tried calling it with shell=False (the default), but
> then I got an error that the file is not a valid Win32 application.
>
> I would appreciate any help with this problem.
>
> -- Timothy
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#72136

Sorry for not being explicit enough. I am aware that this would work if I called python.exe path-to-script.py with shell=False.

In my Python program, I parse an XML file like the one I have included below. Then I loop through the paths of the apps listed in it and run them by calling something like this:

for app_path in app_paths:
    args = shlex.split(app_path.replace("\\", "\\\\"))
    args = [arg.replace("\\\\", "\\") for arg in args]
    args[0] = os.path.expandvars(args[0])
    subprocess.Popen(args, shell='__WXMSW__' in wx.PlatformInfo)

I want users to be able to enter paths in the XML file exactly the way they would be entered in a Windows shortcut. Since it is possible to make a Windows shortcut for path-to-script.py without the python.exe in front of it and have it open in its own command prompt, I want to be able to do the same thing in my XML file, but this is what I cannot figure out.

By the way, is there a simpler way to use shlex.split and return valid Windows paths than the way I am doing (as shown above)?

Thank you.

-- Timothy

*** Contents of app_list.xml below ***

<?xml version='1.0' encoding='utf-8'?>
<apps>
  <app name="Mozilla Firefox">%ProgramFiles%\Mozilla Firefox\firefox.exe</app>
  <app name="Mozilla Thunderbird">%ProgramFiles%\Mozilla Thunderbird\thunderbird.exe</app>
  <app name="LibreOffice Writer">%ProgramFiles%\LibreOffice 4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>
  <app name="**Python script**">C:\Users\Timothy\Documents\Python\myscript.py</app>
</apps>

[toc] | [prev] | [next] | [standalone]

#72139

On 28/05/2014 00:01, ps16thypresenceisfullnessofjoy@gmail.com wrote:

> I want users to be able to enter paths in the XML file exactly the
> way they would be entered in a Windows shortcut. Since it is possible
> to make a Windows shortcut for path-to-script.py without the
> python.exe in front of it and have it open in its own command prompt,
> I want to be able to do the same thing in my XML file, but this is
> what I cannot figure out.

Anything done through shortcuts is making use of the Windows Shell API. 
To mimic that behaviour, you could try using that; in this case, 
ShellExecute[Ex]. For simple purposes, this is exposed in Python as 
os.startfile. If you need more control, you'd have to use the API call 
directly, either via ctypes or via the pywin32 libraries under the 
win32com.shell package.

To mimic the behaviour exactly (if that is a requirement), you could 
actually create a temporary shortcut with the desired information and 
invoke it via os.startfile.

I haven't followed the thread (and I'm offline at the moment) so I'll 
wait until I've seen it before I comment on the shlex.split / \\ dance 
above. On the surface, though, I'm not sure what it's achieving. [All 
right, I didn't wait :)].

TJG

[toc] | [prev] | [next] | [standalone]

#72187

Thank you for your reply.

I think I'll use PyWin32 if it's available on the user's system, and otherwise fall back to using subprocess.Popen, since I want my script to be cross-platform. os.startfile won't work for me because you can't pass arguments to the file being started. (When I first asked my question, I was thinking I might just need to pass a certain STARTUPINFO flag to subprocess.Popen, but it looks like that's not the solution.)

A few examples from the interactive interpreter should help to explain why I am doing the "\ \\ dance" (I used raw strings in these examples so that I wouldn't need to escape the backslashes):

>>> import shlex
>>> shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py')
['C:UsersTimothyDocumentsPythonmyscript.py']
>>> shlex.split(r'C:\\Users\\Timothy\\Documents\\Python\\myscript.py')
['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']
>>> shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py', posix=False)
['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']

The first example shows that single backslashes get removed. The second example shows that double backslashes are preserved intact. The third example shows that if posix=False, single backslashes are converted to double backslashes. None of these three behaviors are acceptable to correctly parse a Windows path, which is why I am doing what I am to work around the issue.

-- Timothy

[toc] | [prev] | [next] | [standalone]

#72183

On 28/05/2014 06:08, Tim Golden wrote:
> On 28/05/2014 00:01, ps16thypresenceisfullnessofjoy@gmail.com wrote:
>
>> I want users to be able to enter paths in the XML file exactly the
>> way they would be entered in a Windows shortcut. Since it is possible
>> to make a Windows shortcut for path-to-script.py without the
>> python.exe in front of it and have it open in its own command prompt,
>> I want to be able to do the same thing in my XML file, but this is
>> what I cannot figure out.
>
> Anything done through shortcuts is making use of the Windows Shell API.
> To mimic that behaviour, you could try using that; in this case,
> ShellExecute[Ex]. For simple purposes, this is exposed in Python as
> os.startfile. If you need more control, you'd have to use the API call
> directly, either via ctypes or via the pywin32 libraries under the
> win32com.shell package.
>
> To mimic the behaviour exactly (if that is a requirement), you could
> actually create a temporary shortcut with the desired information and
> invoke it via os.startfile.
>
> I haven't followed the thread (and I'm offline at the moment) so I'll
> wait until I've seen it before I comment on the shlex.split / \\ dance
> above. On the surface, though, I'm not sure what it's achieving. [All
> right, I didn't wait :)].

I've just read the original post. My answer above isn't quite on the 
nail (although it might still be useful). If you're prepared to use the 
pywin32 libraries, then you could use win32api.FindExecutable to provide 
a first argument to the Popen constructor, followed by the others in 
your <app/> data. Something like this:

<code>
import subprocess
import win32api

# ...
for app_path in app_paths:
     args = ... split ...
     _, exe = win32api.FindExecutable(args[0])
     if exe != os.path.abspath(args[0]):
         args = [exe] + args
     subprocess.call(args)

</code>

As to the shlex dance, I *think* you're trying to break the command line 
up, expand any env vars, and then pass it along to Popen as above?
If your <app/> data were formatted as though for a Windows command-line, 
ie with the paths double-quoted if they contain spaces, then shlex 
should do the right thing by it without any further messing around.

So, if this example:

<app name="LibreOffice Writer">%ProgramFiles%\LibreOffice 
4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>

were instead:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice 
4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>

then the shlex dance would just be:

args = [os.path.expandvars(i) for i in shlex.split(app_path)]

Although, assuming associations were set up in the usual way, the code I 
outlined above to use FindExecutable would cope with this without the 
need to specify the swriter.exe. As would the os.startfile approach I 
suggested earlier.

TJG

[toc] | [prev] | [next] | [standalone]

#72189

Thank you for your replies. I tried what you suggested in your second post and it worked.

That was actually a mistake in the app_list.xml file. As you said:

<app name="LibreOffice Writer">%ProgramFiles%\LibreOffice 4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>

should instead be:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice 4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>

I just made that file as a sample, and didn't actually test it.

My "shlex dance" has nothing to do with that, though. A few examples from the interactive interpreter should explain why I am doing it (I used raw strings in these examples so that I wouldn't need to escape the backslashes):

>>> import shlex
>>> shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py')
['C:UsersTimothyDocumentsPythonmyscript.py']
>>> shlex.split(r'C:\\Users\\Timothy\\Documents\\Python\\myscript.py')
['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']
>>> shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py', posix=False)
['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']

The first example shows that single backslashes get removed. The second example shows that double backslashes are preserved intact. The third example shows that if posix=False, single backslashes are converted to double backslashes. None of these three behaviors are acceptable to correctly parse a Windows path, which is why I am doing what I am to work around the issue.

I think I'll use PyWin32 as you suggested if it's available on the user's system. If it's not, the user will just be required to prefix path-to-script.py with python.exe. (When I first asked my question, I was thinking I might just need to pass a certain STARTUPINFO flag to subprocess.Popen, but it looks like that's not the solution.)

-- Timothy

[toc] | [prev] | [next] | [standalone]

#72235

On 28/05/2014 21:46, ps16thypresenceisfullnessofjoy@gmail.com wrote:
> 
> Thank you for your replies. I tried what you suggested in your second
> post and it worked.
> 
> That was actually a mistake in the app_list.xml file. As you said:
> 
> <app name="LibreOffice Writer">%ProgramFiles%\LibreOffice
> 4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>
> 
> should instead be:
> 
> <app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice
> 4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>
> 
> I just made that file as a sample, and didn't actually test it.
> 
> My "shlex dance" has nothing to do with that, though. A few examples
> from the interactive interpreter should explain why I am doing it (I
> used raw strings in these examples so that I wouldn't need to escape
> the backslashes):
> 
>>>> import shlex 
>>>> shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py')
> ['C:UsersTimothyDocumentsPythonmyscript.py']
>>>> shlex.split(r'C:\\Users\\Timothy\\Documents\\Python\\myscript.py')
>
>>>> 
['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']
>>>> shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py',
>>>> posix=False)
> ['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']
> 
> The first example shows that single backslashes get removed. The
> second example shows that double backslashes are preserved intact.
> The third example shows that if posix=False, single backslashes are
> converted to double backslashes. None of these three behaviors are
> acceptable to correctly parse a Windows path, which is why I am doing
> what I am to work around the issue.

Well I certainly learnt something there! An additional test, which you
don't show is this:

>>> import shlex
>>> shlex.split(r'"c:\users\timothy\documents"')
['c:\\users\\timothy\\documents']
>>>

In other words, given the double-quoted data in your XML file, I think
it will do the right thing by Windows backslashes. YMMV, I suppose.

TJG

[toc] | [prev] | [next] | [standalone]

#72248

That's interesting, now I learned something else too. As I said before, though, I want users to be able to enter paths in the XML file exactly the way they would be entered in a Windows shortcut.
(Actually, my program currently only has one Windows user, for whom I develop it [I don't even use it myself :-0].)

Since in a Windows shortcut you don't need to put quotes around a path that doesn't contain spaces, I want to follow that behavior in my program as well.

So actually, I see that the line:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice 4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>

should instead be:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice 4\program\swriter.exe" C:\Users\Timothy\Documents\myfile.odt</app>

because there are no spaces in the second path, but there is a space in the first one.

So I guess I'll still have to do my "shlex dance" (which for some reason I keep on mistyping as "shlex dane" :-)) unless I can learn a better way to do it.

Also, is my calling os.path.expandvars only on the first argument a good idea? I want to accept environment variables in my program, but I don't want to call os.path.expandvars on all the arguments, because I want to let other programs take care of parsing their own arguments.

I'm just wondering if I should relook at the whole way that I am doing things here. Maybe it would be better to just use shlex.split and subprocess.Popen and not try to follow the behavior of Windows shortcuts so closely? (But then it might be hard for some Windows users to understand how to use the program if I didn't.) What do you think?

[toc] | [prev] | [next] | [standalone]

#72377

29/05/2014 20:21, ps16thypresenceisfullnessofjoy@gmail.com wrote:
> That's interesting, now I learned something else too. As I said
> before, though, I want users to be able to enter paths in the XML
> file exactly the way they would be entered in a Windows shortcut.

[...]

> Since in a Windows shortcut you don't need to put quotes around a
> path that doesn't contain spaces, I want to follow that behavior in
> my program as well.

[...]

> So I guess I'll still have to do my "shlex dance" (which for some
> reason I keep on mistyping as "shlex dane" :-)) unless I can learn a
> better way to do it.

It looks like it. Sometimes the behaviour out of the box just doesn't 
cut it. Don't forget, also, that you can subclass, eg, the shlex.shlex 
class and tweak it to do what you need.

>
> Also, is my calling os.path.expandvars only on the first argument a
> good idea? I want to accept environment variables in my program, but
> I don't want to call os.path.expandvars on all the arguments, because
> I want to let other programs take care of parsing their own
> arguments.

I'd say that's a call only you can make given what you know of your 
requirements. That said, I'm not sure what the "other programs" are 
going to do with any embedded env vars except to expand them as you're 
doing.

Ultimately, don't tie yourselves in knots either trying too hard to 
emulate Windows shortcuts or trying too hard to use shlex.split out of 
the box. Decide what constitutes a useful combination of features and 
implement them as simply as you can. [He says, winning the prize for 
stating the obvious :) ]

TJG

[toc] | [prev] | [next] | [standalone]

#72660

Thanks again for your help. I tried something similar to what you suggested:

def run_app(self, app_path):
    args = shlex.split(app_path.replace("\\", "\\\\"))
    args = [arg.replace("\\\\", "\\") for arg in args]
    args[0] = os.path.expandvars(args[0])
    try:
        if pywin32:
            exe = win32api.FindExecutable(args[0])[1]
            if exe != os.path.abspath(args[0]):
                args.insert(0, exe)
        subprocess.Popen(args)
    except Exception:
        return False
    return True

where pywin32 is a global variable defined earlier in my program based on whether or not the win32api module is available.
This code generally works quite well, but with one problem. I have in my XML file a line like this:

<app name="Notepad++" checked="false">%ProgramFiles%\Notepad++\notepad++.exe C:\Users\Timothy\Documents\Python\StartupPlus\apps.xml</app>

Within the try statement, the variable 'args' contains this list, as expected:

['C:\Program Files (x86)\Notepad++\notepad++.exe', 'C:\Users\Timothy\Documents\Python\StartupPlus\apps.xml']

But the 'exe' variable returned by win32api.FindExecutable is not what I would expect:

C:\PROGRA~2\NOTEPA~1\NOTEPA~1.EXE

Because of this, the boolean comparison:

if exe != os.path.abspath(args[0]):

returns True when it should return False, since it is comparing a short (MS-DOS) path name with a long path name.
Should I use win32file.GetLongPathName to fix this problem, or do you know of a better solution?

-- Timothy

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web