Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #23473 > unrolled thread

exec problem is JDK 1.7.0_21

Back to article view | Back to comp.lang.java.programmer

Contents

  exec problem is JDK 1.7.0_21 Roedy Green <see_website@mindprod.com.invalid> - 2013-04-16 15:48 -0700
    Re: exec problem is JDK 1.7.0_21 Lew <lewbloch@gmail.com> - 2013-04-16 16:48 -0700
    Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-20 13:00 +0300
    Re: exec problem is JDK 1.7.0_21 Joerg Meier <joergmmeier@arcor.de> - 2013-04-20 15:13 +0200
      Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-20 19:07 +0300
        Re: exec problem is JDK 1.7.0_21 markspace <markspace@nospam.nospam> - 2013-04-20 09:24 -0700
          Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-20 19:34 +0300
            Re: exec problem is JDK 1.7.0_21 markspace <markspace@nospam.nospam> - 2013-04-20 11:01 -0700
              Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 00:25 +0300
            Re: exec problem is JDK 1.7.0_21 Joerg Meier <joergmmeier@arcor.de> - 2013-04-20 22:03 +0200
              Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 00:29 +0300
                Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-20 21:56 +0000
                  Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 08:31 +0300
                    Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-21 11:08 +0000
                      Re: exec problem is JDK 1.7.0_21 Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-04-21 13:43 -0300
                      Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 23:12 +0300
                        Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-21 21:17 +0000
                        Re: exec problem is JDK 1.7.0_21 Arne Vajhøj <arne@vajhoej.dk> - 2013-04-27 22:49 -0400
                Re: exec problem is JDK 1.7.0_21 Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-04-20 15:03 -0700
                  Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 08:26 +0300
                    Re: exec problem is JDK 1.7.0_21 "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-04-21 14:30 +0100
                      Re: exec problem is JDK 1.7.0_21 Knute Johnson <nospam@knutejohnson.com> - 2013-04-21 09:20 -0700
                        Re: exec problem is JDK 1.7.0_21 markspace <markspace@nospam.nospam> - 2013-04-21 11:31 -0700
                          Re: exec problem is JDK 1.7.0_21 Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-04-21 16:06 -0300
                            Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-21 19:47 +0000
                              Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-21 23:01 +0000
                        Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 23:29 +0300
                          Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 23:45 +0300
                          Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-21 21:38 +0000
                          Re: exec problem is JDK 1.7.0_21 Knute Johnson <nospam@knutejohnson.com> - 2013-04-21 18:49 -0700
                            Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-22 09:36 +0300
                              Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-22 10:39 +0100
                                Re: exec problem is JDK 1.7.0_21 Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-04-22 07:43 -0700
                                  Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-22 19:33 +0300
                                    Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-23 07:30 +0100
                            Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-22 21:07 +0000
                              Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-23 23:28 +0300
                                Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-23 21:05 +0000
                                  Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-24 11:22 +0300
                                    Re: exec problem is JDK 1.7.0_21 Martin Gregorie <martin@address-in-sig.invalid> - 2013-04-24 20:22 +0000
                                  Re: exec problem is JDK 1.7.0_21 Nigel Wade <nmw@ion.le.ac.uk> - 2013-04-24 11:24 +0100
                      Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 23:23 +0300
              Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-21 00:40 +0300
                Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-23 10:48 +0100
                  Re: exec problem is JDK 1.7.0_21 Donkey Hottie <donkey@fredriksson.dy.fi> - 2013-04-23 13:31 +0300
                    <? extends String> (was: exec problem is JDK 1.7.0_21) Steven Simpson <ss@domain.invalid> - 2013-04-23 14:55 +0100
                      Re: <? extends String> lipska the kat <"nospam at neversurrender dot co dot uk"> - 2013-04-23 16:37 +0100
                        Re: <? extends String> Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-04-23 08:50 -0700
                          Re: <? extends String> lipska the kat <"nospam at neversurrender dot co dot uk"> - 2013-04-23 17:30 +0100
                            Re: <? extends String> Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-04-23 09:58 -0700
                              Re: <? extends String> Donkey Hottie <donkey@fredriksson.dy.fi> - 2013-04-23 21:06 +0300
                              Re: <? extends String> lipska the kat <"nospam at neversurrender dot co dot uk"> - 2013-04-23 19:33 +0100
                                Re: <? extends String> Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-04-23 12:26 -0700
                                  Re: <? extends String> lipska the kat <"nospam at neversurrender dot co dot uk"> - 2013-04-24 09:08 +0100
                          Re: <? extends String> Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-04-23 16:28 -0300
                      Re: <? extends String> (was: exec problem is JDK 1.7.0_21) Lew <lewbloch@gmail.com> - 2013-04-23 14:12 -0700
                        Re: <? extends String> Steven Simpson <ss@domain.invalid> - 2013-04-23 23:22 +0100
                  Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-23 19:56 +0100
                    Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-23 23:31 +0300
                      Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-23 23:39 +0100
                        Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-24 09:29 +0300
                          Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-24 10:26 +0100
                            Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-24 13:11 +0300
                              Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-24 22:24 +0100
                                Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-25 11:49 +0300
                                  Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-25 10:19 +0100
                            Re: exec problem is JDK 1.7.0_21 "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-04-25 09:02 +0100
                              Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-25 11:47 +0300
                              Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-25 09:50 +0100
                                Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-25 10:07 +0100
                                  Re: exec problem is JDK 1.7.0_21 "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-04-27 08:44 +0100
                            Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-26 20:17 +0300
                              Re: exec problem is JDK 1.7.0_21 Steven Simpson <ss@domain.invalid> - 2013-04-27 09:22 +0100
                                Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-27 12:39 +0300
                  Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-23 23:17 +0300
                    Re: exec problem is JDK 1.7.0_21 Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-04-23 17:32 -0300
                      Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-24 10:39 +0300
                        Re: exec problem is JDK 1.7.0_21 "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-04-25 09:02 +0100
                          Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-25 11:59 +0300
                            Re: exec problem is JDK 1.7.0_21 Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-04-25 18:38 -0300
                              Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-28 10:57 +0300
                        Re: exec problem is JDK 1.7.0_21 Arne Vajhøj <arne@vajhoej.dk> - 2013-04-27 22:35 -0400
                          Re: exec problem is JDK 1.7.0_21 Sven Köhler <remove-sven.koehler@gmail.com> - 2013-04-28 09:23 +0300
                            Re: exec problem is JDK 1.7.0_21 Arne Vajhøj <arne@vajhoej.dk> - 2013-04-28 09:52 -0400
    Re: exec problem is JDK 1.7.0_21 Stanimir Stamenkov <s7an10@netscape.net> - 2013-04-20 17:52 +0300
      Re: exec problem is JDK 1.7.0_21 Arne Vajhøj <arne@vajhoej.dk> - 2013-04-27 22:31 -0400

Page 4 of 5 — ← Prev page 1 2 3 [4] 5  Next page →

#23611

Am 24.04.2013 01:39, schrieb Steven Simpson:
>> Also, you have to add another backslash if the quote does not terminate
>> the argument.
> 
> Again, I took it to mean that the first two rules produce the same
> result, so \\\" and \\" both produce \".

The documentation of CommandLineToArgv is incomplete. \\" produces a
single backslash, and the quote is the closing quote.
\" and \\\" produce a non-closing quote resp. a backslash followed by a
non-closing quote.

>> Here's my attempt to the do the escaping:
>> http://sourceforge.net/p/lejos/code/HEAD/tree/trunk/org.lejos.nxt.ldt/src/main/java/org/lejos/nxt/ldt/util/LeJOSNXJUtil.java#l421
>>
> 
> I haven't looked in detail at it, but do note your comment:
>> * How decoding works:
>> * 2n backslashes + quote => n backslashes + closing quote
>> * 2n+1 backslashes + quote => n backslashes + inner quote
>> * n backslashes not followed by a quote => n backslashes
> 
> From
> <http://msdn.microsoft.com/en-us/library/windows/desktop/bb776391%28v=vs.85%29.aspx>
> (with my emphasis):
>>
>>   * 2/n/ backslashes followed by a quotation mark produce /n/
>>     backslashes followed by a quotation mark.
>>   * (2/n/) + 1 backslashes followed by a quotation mark *again*
>>     produce /n/ backslashes followed by a quotation mark.
>>   * /n/ backslashes not followed by a quotation mark simply produce
>>     /n/ backslashes.
>>
>>
> 
> Where are you getting the notion that the first two rules imply
> different quotes?  I interpret both as being literal (inner, right?).

Well, by testing. As you can test yourself on the command line, \\" will
result in a closing quote and \" and \\\" will result in non-closing quotes.

Also, there HAS to be a way to distinguish a closing vs. a non-closing
quote. Otherwise you wouldn't know whether the quotes in \" \\" and \\\"
were closing or not. That's it is not documented is just a real shame!


Regards,
  Sven

[toc] | [prev] | [next] | [standalone]

#23618

On 24/04/13 07:29, Sven Köhler wrote:
> Am 24.04.2013 01:39, schrieb Steven Simpson:
>>> Also, you have to add another backslash if the quote does not terminate
>>> the argument.
>> Again, I took it to mean that the first two rules produce the same
>> result, so \\\" and \\" both produce \".
> The documentation of CommandLineToArgv is incomplete.

Agreed.

>> Where are you getting the notion that the first two rules imply
>> different quotes?  I interpret both as being literal (inner, right?).
> Well, by testing. As you can test yourself on the command line, \\" will
> result in a closing quote and \" and \\\" will result in non-closing quotes.

That's not what I'm seeing.  In fact, backslashes appear to have no effect!

Cross-compiled with mingw32:

#include <windows.h>
#include <shellapi.h>

#include <stdio.h>
#include <wchar.h>

static void test(const wchar_t *s)
{
   printf("\nInput line: %ls\n", s);
   int argc;
   LPWSTR *argv = CommandLineToArgvW(s, &argc);
   if (!argv) {
     printf("  unable to parse\n");
   } else {
     printf("  arg count: %d\n", argc);
     for (int i = 0; i < argc; i++)
       printf("  [%d]=[%ls]\n", i, argv[i]);
     LocalFree(argv);
   }
}

int main(void)
{
   test(L"foo");
   test(L"foo\"");
   test(L"foo\\\"");
   test(L"foo\\\\\"");
   test(L"foo\\\\\\\"");

   test(L"\"foo");
   test(L"\"foo\"");
   test(L"\"foo\\\"");
   test(L"\"foo\\\\\"");
   test(L"\"foo\\\\\\\"");

   test(L"foo bar");
   test(L"foo\"bar");
   test(L"foo\\\"bar");
   test(L"foo\\\\\"bar");
   test(L"foo\\\\\\\"bar");
  
   test(L"foo\\ bar");
   test(L"foo\\\\ bar");
   test(L"foo\\\\\\ bar");

   test(L"\"foo bar\"");
   test(L"\"foo \\\"bar\"");
   test(L"\"foo \\\\\"bar\"");
   test(L"\"foo \\\\\\\"bar\"");

   return 0;
}




Output:

Input line: foo
   arg count: 1
   [0]=[foo]

Input line: foo"
   arg count: 1
   [0]=[foo"]

Okay, so if the quote doesn't start the argument, it's literal.


Input line: foo\"
   arg count: 1
   [0]=[foo\"]

Input line: foo\\"
   arg count: 1
   [0]=[foo\\"]

Input line: foo\\\"
   arg count: 1
   [0]=[foo\\\"]

These backslashes aren't folded in any way.

Input line: "foo
   arg count: 1
   [0]=[foo]

Input line: "foo"
   arg count: 1
   [0]=[foo]

So the leading quote is stripped, and the closing quote is optional - no 
error.


Input line: "foo\"
   arg count: 1
   [0]=[foo\]

Input line: "foo\\"
   arg count: 1
   [0]=[foo\\]

Input line: "foo\\\"
   arg count: 1
   [0]=[foo\\\]

Inside the quoted argument, backslashes still have no special meaning.  
And I can't get a literal quote character.


Input line: foo bar
   arg count: 2
   [0]=[foo]
   [1]=[bar]

So a space splits arguments.


Input line: foo"bar
   arg count: 1
   [0]=[foo"bar]

The quote inside an unquoted argument is taken as literal.


Input line: foo\"bar
   arg count: 1
   [0]=[foo\"bar]

Input line: foo\\"bar
   arg count: 1
   [0]=[foo\\"bar]

Input line: foo\\\"bar
   arg count: 1
   [0]=[foo\\\"bar]

The backslashes are literal in the middle of the unquoted argument.


Input line: foo\ bar
   arg count: 2
   [0]=[foo\]
   [1]=[bar]

Input line: foo\\ bar
   arg count: 2
   [0]=[foo\\]
   [1]=[bar]

Input line: foo\\\ bar
   arg count: 2
   [0]=[foo\\\]
   [1]=[bar]

Just checking that there's no unspecified way to escape a space.


Input line: "foo bar"
   arg count: 1
   [0]=[foo bar]

The quotes cause the space to be taken literally.


Input line: "foo \"bar"
   arg count: 2
   [0]=[foo \]
   [1]=[bar]

Input line: "foo \\"bar"
   arg count: 2
   [0]=[foo \\]
   [1]=[bar]

Input line: "foo \\\"bar"
   arg count: 2
   [0]=[foo \\\]
   [1]=[bar]

Backslashes are still literal, and the next quote closes the argument 
regardless.  Plus, the extra quote after [bar] is not literal.

There seems to be no way to escape a space other than enclosing the 
entire argument in quotes.  Once inside the quotes, there's no way to 
escape anything else (like a literal quote), so there's no way to escape 
an argument containing a space and a quote.  Backslashes behave 
literally everywhere.

Is the program using CommandLineToArgvW incorrectly?  Are there any 
other input strings to try?



> Also, there HAS to be a way to distinguish a closing vs. a non-closing
> quote. Otherwise you wouldn't know whether the quotes in \" \\" and \\\"
> were closing or not. That's it is not documented is just a real shame!

What I'm seeing now is that general escaping is impossible, not merely 
documented badly.  :-(


-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#23619

On 04/24/2013 12:26 PM, Steven Simpson wrote:
> Input line: "foo \"bar"
>    arg count: 2
>    [0]=[foo \]
>    [1]=[bar]
>
> Input line: "foo \\"bar"
>    arg count: 2
>    [0]=[foo \\]
>    [1]=[bar]
>
> Input line: "foo \\\"bar"
>    arg count: 2
>    [0]=[foo \\\]
>    [1]=[bar]
>
> Backslashes are still literal, and the next quote closes the argument
> regardless.  Plus, the extra quote after [bar] is not literal.

That's completely not what the documentation says. The documentation 
clearly states, that 2n or 2n+1 backslashes followed by a quote should 
result in backslashes. However, in your cases, not even the number of 
backslashes matches the documentation.

Which operating system were you using? I will try to check tonight on 
Windows 7 using mingw64. Are you sure, you haven't mixed char and wchar 
strings? Have you tried CommandLineToArgvA?


Regards,
   Sven

[toc] | [prev] | [next] | [standalone]

#23633

On 24/04/13 11:11, Sven Köhler wrote:
> Which operating system were you using? 

winver reports "Windows 7 Enterprise".  On a VirtualBox VM.

> I will try to check tonight on Windows 7 using mingw64. Are you sure, 
> you haven't mixed char and wchar strings? 

I don't think so.  If I had, I wouldn't expect anything coherent.

> Have you tried CommandLineToArgvA?

Does such a beast exist?  Googling, it seems to be wishware.


-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#23641

On 04/25/2013 12:24 AM, Steven Simpson wrote:
> On 24/04/13 11:11, Sven Köhler wrote:
>> I will try to check tonight on Windows 7 using mingw64. Are you sure,
>> you haven't mixed char and wchar strings?
>
> I don't think so.  If I had, I wouldn't expect anything coherent.

Let's forget about CommandLineToArgvW. It's broken (doesn't implement 
what is documented) and it doesn't distinguish between inner and outer 
quotes.

The proper documentation about how microsoft (and apperently mingw32 and 
mingw64) do the command line tokenization is available here:

http://msdn.microsoft.com/en-us/library/a1y7w461.aspx


Regards,
   Sven

[toc] | [prev] | [next] | [standalone]

#23645

On 25/04/13 09:49, Sven Köhler wrote:
> On 04/25/2013 12:24 AM, Steven Simpson wrote:
>> On 24/04/13 11:11, Sven Köhler wrote:
>>> I will try to check tonight on Windows 7 using mingw64. Are you sure,
>>> you haven't mixed char and wchar strings?
>>
>> I don't think so.  If I had, I wouldn't expect anything coherent.
>
> Let's forget about CommandLineToArgvW. It's broken (doesn't implement 
> what is documented) and it doesn't distinguish between inner and outer 
> quotes.

Agreed.

> The proper documentation about how microsoft (and apperently mingw32 
> and mingw64) do the command line tokenization is available here:
>
> http://msdn.microsoft.com/en-us/library/a1y7w461.aspx

Good.

-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#23639

Steven Simpson wrote:

> Cross-compiled with mingw32:
...
> int main(void)

What is doing the argument splitting here, and where is its spec ?

Point is that when you start an executable on Windows /you do not pass an array
of strings/ to the OS.  You pass a single string (or rather two strings -- one
to name the executable and the other to be the /entire/ command line).

That means that the interpretation of that (single) string as an array of
sub-strings is /entirely/ at the target application's discretion.  And hence
entirely arbitrary.   Arbitrary in practise I mean, not just in theory.  (I
have a micro-application in production that recognises quotes around its first
argument but not around its second ;-)

/Some/ applications use the built-in CommandLineToArgvW() function in Windows,
but most (at least as far as my knowledge goes) do not.  And anyway that
function's defined behaviour defies belief (clearly the documentation simply
enshrines the existing behaviour of some seriously stupid code).
ProcessBuilder (if we take the code rather than the documentation as the spec)
is a disgrace (if we just take the doc as the spec then it isn't even that).

I have no idea what command-line parser mingw32 provides for (imposes on) the
code it compiles.  It /may/ be the same as CommandLineToArgvW() (or even be
implemented using it), but my guess is that the mingw folk will have come up
with something closer to what /bin/sh does in *nix.

    -- chris

[toc] | [prev] | [next] | [standalone]

#23640

On 04/25/2013 11:02 AM, Chris Uppal wrote:
> Steven Simpson wrote:
>
>> Cross-compiled with mingw32:
> ...
>> int main(void)
>
> What is doing the argument splitting here, and where is its spec ?

Here's the spec of what microsoft implements:
[1] http://msdn.microsoft.com/en-us/library/a1y7w461.aspx

> Point is that when you start an executable on Windows /you do not pass an array
> of strings/ to the OS.  You pass a single string (or rather two strings -- one
> to name the executable and the other to be the /entire/ command line).

Exactly!

> That means that the interpretation of that (single) string as an array of
> sub-strings is /entirely/ at the target application's discretion.  And hence
> entirely arbitrary.   Arbitrary in practise I mean, not just in theory.  (I
> have a micro-application in production that recognises quotes around its first
> argument but not around its second ;-)

And hence the current ProcessBuilder behavior is simply broken.

> /Some/ applications use the built-in CommandLineToArgvW() function in Windows,

I'm afraid, that has turned out to be false information. Executable 
generated by Microsoft compilers seem to use some other code, that 
follows the spec given in [1], but CommandLineToArgvW does not follow 
this specification. In fact, CommandLineToArgvW doesn't do anything 
useful. Let's forget I ever mentioned it.

> but most (at least as far as my knowledge goes) do not.  And anyway that
> function's defined behaviour defies belief (clearly the documentation simply
> enshrines the existing behaviour of some seriously stupid code).
> ProcessBuilder (if we take the code rather than the documentation as the spec)
> is a disgrace (if we just take the doc as the spec then it isn't even that).

Yes.

> I have no idea what command-line parser mingw32 provides for (imposes on) the
> code it compiles.  It /may/ be the same as CommandLineToArgvW() (or even be
> implemented using it), but my guess is that the mingw folk will have come up
> with something closer to what /bin/sh does in *nix.

I would hope, that mingw32 and mingw64 toolchains have code that is 
compatible to [1]. At least my experience with mingw and visual studio 
executables is consistent with [1].


Regards,
   Sven

[toc] | [prev] | [next] | [standalone]

#23643

On 25/04/13 09:02, Chris Uppal wrote:
> Steven Simpson wrote:
>
>> Cross-compiled with mingw32:
> ...
>> int main(void)
> What is doing the argument splitting here, and where is its spec ?

Um, the call to CommandLineToArgvW here:

static void test(const wchar_t *s)
{
   printf("\nInput line: %ls\n", s);
   int argc;
   LPWSTR *argv = CommandLineToArgvW(s, &argc);
   if (!argv) {
     printf("  unable to parse\n");
   } else {
     printf("  arg count: %d\n", argc);
     for (int i = 0; i < argc; i++)
       printf("  [%d]=[%ls]\n", i, argv[i]);
     LocalFree(argv);
   }
}


The program itself takes no arguments.  All "command lines" are 
hard-coded into the program to avoid any possibility of being mangled by 
any hidden parser (whether it's in the invoking shell or in the program 
before main() is invoked).  Since the command lines are embedded in the 
program, the only escaping we need to apply to them is as for C strings, 
which is well understood.  The program even prints out both the original 
command line and the resulting argument list, so we can see both input 
and output with even the C-string escaping removed.


The 'spec' for CommandLineToArgvW:

<http://msdn.microsoft.com/en-us/library/windows/desktop/bb776391%28v=vs.85%29.aspx>


-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#23644

On 25/04/13 09:50, Steven Simpson wrote:
> On 25/04/13 09:02, Chris Uppal wrote:
>> Steven Simpson wrote:
>>
>>> Cross-compiled with mingw32:
>> ...
>>> int main(void)
>> What is doing the argument splitting here, and where is its spec ?
>
> Um,[...]

Sorry, I think I just answered a question you weren't asking, now I've 
seen Sven's response...  :-/

Um.

-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#23665

Steven Simpson wrote:

> Sorry, I think I just answered a question you weren't asking, now I've
> seen Sven's response...  :-/

No, my mistake.  I didn't read you actual code carefully enough to see that you 
were making a different poiint from what I'd assumed.

    -- chris 

[toc] | [prev] | [next] | [standalone]

#23656

Am 24.04.2013 12:26, schrieb Steven Simpson:
> There seems to be no way to escape a space other than enclosing the
> entire argument in quotes.  Once inside the quotes, there's no way to
> escape anything else (like a literal quote), so there's no way to escape
> an argument containing a space and a quote.  Backslashes behave
> literally everywhere.
> 
> Is the program using CommandLineToArgvW incorrectly?  Are there any
> other input strings to try?

I tested
  test(L"foo.exe \"ab\\\"c\\\\\" def");
  test(L"\"foo.exe\" \"ab\\\"c\\\\\" def");
end then it works. Apparently, the first token (which is the program
name, and can be assumed not to contain any backslashes and quotes) is
treated differently from all subsequent tokens.

So CommandLineToArgvW does allow for escaping quotes within the arguments.


Regards,
  Sven

[toc] | [prev] | [next] | [standalone]

#23667

On 26/04/13 18:17, Sven Köhler wrote:
> Am 24.04.2013 12:26, schrieb Steven Simpson:
>> Is the program using CommandLineToArgvW incorrectly?  Are there any
>> other input strings to try?
> I tested
>    test(L"foo.exe \"ab\\\"c\\\\\" def");
>    test(L"\"foo.exe\" \"ab\\\"c\\\\\" def");
> end then it works.

Confirmed.  I tried all my cases with a prefixed token, and they started 
working too.


>   Apparently, the first token (which is the program
> name, and can be assumed not to contain any backslashes and quotes) is
> treated differently from all subsequent tokens.

Well, the first token can contain backslashes, but they are taken literally.


> So CommandLineToArgvW does allow for escaping quotes within the arguments.

I take it all back.


-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#23668

Am 27.04.2013 11:22, schrieb Steven Simpson:
> On 26/04/13 18:17, Sven Köhler wrote:
>>   Apparently, the first token (which is the program
>> name, and can be assumed not to contain any backslashes and quotes) is
>> treated differently from all subsequent tokens.
> 
> Well, the first token can contain backslashes, but they are taken
> literally.

I should haven written "can be assumed not to contain any quotes and
backslashes followed by a quote".

[toc] | [prev] | [next] | [standalone]

#23597

Am 23.04.2013 12:48, schrieb Steven Simpson:
> On 20/04/13 22:40, Sven Köhler wrote:
>> Oh, and of course the ProcessBuilder doesn't behave as it is supposed
>> to. As mentioned in my first post, the String "\"a b\"" would be passed
>> unmodified to the program invoked. However, clearly, the string passed
>> to the program should have been
>> "\"\\\"a b\\\"\""
>>
>> Only with the quotes and backslashes added, CommandLineToArgv would
>> decode it to "\"a b\"". With the current ProcessBuilder implementation,
>> a Windows program will see the parameter "a b" while on UNIX the program
>> will see "\"a b\"".
>>
>> See
>> http://msdn.microsoft.com/en-us/library/windows/desktop/bb776391%28v=vs.85%29.aspx
>>
>> for details.
> 
> You're expecting Java to build its Windows command string something like
> this?:
> ...code..
> 
> Does that work correctly for anything to be thrown at CommandLineToArgvW?

I haven't checked your code, but I have written the inverse to
CommandLineToArgv in Java. It's not that hard. Passing the result to
ProcessBuilder works fine - even though that it works is based on the
two silly undocumented facts: (1) ProcessBuilder adds quotes if and only
if the argument doesn't start and end with quotes and (2) ProcessBuilder
doesn't mess with quotes and backslashes inside the arguments.

The problem with Java's ProcessBuilder is, that people use it "willy
nilly" being completely oblivous about the fact that it doesn't do what
might be best for them. (Recall the case "c:\\program files\\", "world"
where the two strings are basically merged to "c:\\program files\"
world". This is probably the scariest.)
Secondly, there is no way to pass the arguments correctly by only
relying on documented stuff.

>> Note, that I assume that the program invoked uses CommandLineToArgv to
>> decode the command line. Which is by no means clear, as any program can
>> implement their own tokenizer.
>>
>> Don't you find that a bit strange?
> 
> Perhaps that assumption is too risky for Java to make, e.g. there are
> enough 'native' Windows/DOS commands around that the programmer is
> likely to want to invoke, but don't use CommandLineToArgvW, and so would
> be confused if they received a string escaped as above. Not a very
> satisfactory situation.

Some assumption about the program's tokenizer are already part of
ProcessBuilder's implementation (namely that they understand quotes
around arguments with spaces). Isn't that risky already?

It still puzzles me: The documentation explicitly mentions operating
systems, where programs perform the tokenization themselves. It even
suggests that the String-list should exist of exactly two elements
(which could maybe interpretated as a hint that the second element could
be the raw command line parameter string). But instead, what we get, is
an imperfect, error prone, undocumented mangling of arguments: adding
quotes without taking care of the quotes within the strings is
something, that shouldn't slip any programmer's attention.


Regards,
  Sven

[toc] | [prev] | [next] | [standalone]

#23600

On 04/23/2013 05:17 PM, Sven Köhler wrote:
> Am 23.04.2013 12:48, schrieb Steven Simpson:
>> On 20/04/13 22:40, Sven Köhler wrote:
>>> Oh, and of course the ProcessBuilder doesn't behave as it is supposed
>>> to. As mentioned in my first post, the String "\"a b\"" would be passed
>>> unmodified to the program invoked. However, clearly, the string passed
>>> to the program should have been
>>> "\"\\\"a b\\\"\""
>>>
>>> Only with the quotes and backslashes added, CommandLineToArgv would
>>> decode it to "\"a b\"". With the current ProcessBuilder implementation,
>>> a Windows program will see the parameter "a b" while on UNIX the program
>>> will see "\"a b\"".
>>>
>>> See
>>> http://msdn.microsoft.com/en-us/library/windows/desktop/bb776391%28v=vs.85%29.aspx
>>>
>>> for details.
>>
>> You're expecting Java to build its Windows command string something like
>> this?:
>> ...code..
>>
>> Does that work correctly for anything to be thrown at CommandLineToArgvW?
>
> I haven't checked your code, but I have written the inverse to
> CommandLineToArgv in Java. It's not that hard. Passing the result to
> ProcessBuilder works fine - even though that it works is based on the
> two silly undocumented facts: (1) ProcessBuilder adds quotes if and only
> if the argument doesn't start and end with quotes and (2) ProcessBuilder
> doesn't mess with quotes and backslashes inside the arguments.
>
> The problem with Java's ProcessBuilder is, that people use it "willy
> nilly" being completely oblivous about the fact that it doesn't do what
> might be best for them. (Recall the case "c:\\program files\\", "world"
> where the two strings are basically merged to "c:\\program files\"
> world". This is probably the scariest.)
> Secondly, there is no way to pass the arguments correctly by only
> relying on documented stuff.
[ SNIP ]

That's the point a number of us have been making, Sven. In the case of 
ProcessBuilder and Process you jump to no conclusions, and test your 
specific case. Operative word being "test", as in unit test, functional 
test, integration test, system test, UAT.

AHS

[toc] | [prev] | [next] | [standalone]

#23613

On 04/23/2013 11:32 PM, Arved Sandstrom wrote:
> That's the point a number of us have been making, Sven. In the case of
> ProcessBuilder and Process you jump to no conclusions, and test your
> specific case.

The specific case is broken. Now what?

[toc] | [prev] | [next] | [standalone]

#23638

Sven Köhler wrote:

> The specific case is broken. Now what?

Then quite possibly you've hit the end of the road.

Perhaps you can get around the limits of ProcessBuilder by (say) encoding the 
desired command-line in base64 and using a helper application which will decode 
it before passing it to Windows, but even with hacks like that, there's no 
guarantee that you can pass arbitrary arguments to an arbitrary application.

Depending on how its command-line parser is written it might be simply 
impossible to pass, say, a single double-quote as an argument no matter /what/ 
string you pass to Windows as the command-line.

    -- chris 

[toc] | [prev] | [next] | [standalone]

#23642

On 04/25/2013 11:02 AM, Chris Uppal wrote:
> Sven Köhler wrote:
>> The specific case is broken. Now what?
>
> Then quite possibly you've hit the end of the road.

Which should imply, that ProcessBuilder is currently broken on Windows.
(Or as you said it: On Windows, ProcessBuilder is a disgrace)

I was just hoping for a comment of Arved, since I really didn't 
understand the point of "let's test a specific case" while my point was 
"the general idea behind ProcessBuilder is broken, in case the OS is 
Windows".

> Perhaps you can get around the limits of ProcessBuilder by (say) encoding the
> desired command-line in base64 and using a helper application which will decode
> it before passing it to Windows, but even with hacks like that, there's no
> guarantee that you can pass arbitrary arguments to an arbitrary application.

Passing arbitrary arguments to arbitrary applications is not always 
possible on Windows - which is a shame! However: assuming I know 
everything about the application I am calling, ProcessBuilder is not a 
big help in calling that application as it messes around with the 
command line parameters. My point is: on Windows, it shouldn't do so. 
Actually, on Windows, it should pass the second element of the String 
list as-is - without messing with it - and throw an exception if the 
String list contains more than 2 elements.

For people that want escaping compatible with microsoft's default 
tokenization, ProcessBuilder should provide some helper function. 
ProcessBuilder should also provide some function to determine, what kind 
of OS you're dealing with, as the documentation, which contains 
statements like "on some operating systems" is not helping _at all_.

> Depending on how its command-line parser is written it might be simply
> impossible to pass, say, a single double-quote as an argument no matter /what/
> string you pass to Windows as the command-line.

That is true. But with ProcessBuilder it is in general impossible to 
adapt to whatever command-line parser is implemented. That's just a 
shame, considering that ProcessBuilder is not that old.


Regards,
   Sven

[toc] | [prev] | [next] | [standalone]

#23650

On 04/25/2013 05:59 AM, Sven Köhler wrote:
> On 04/25/2013 11:02 AM, Chris Uppal wrote:
>> Sven Köhler wrote:
>>> The specific case is broken. Now what?
>>
>> Then quite possibly you've hit the end of the road.
>
> Which should imply, that ProcessBuilder is currently broken on Windows.
> (Or as you said it: On Windows, ProcessBuilder is a disgrace)
>
> I was just hoping for a comment of Arved, since I really didn't
> understand the point of "let's test a specific case" while my point was
> "the general idea behind ProcessBuilder is broken, in case the OS is
> Windows".
>
[ SNIP ]

My comment just meant what it said - identify the specific app on 
Windows, the arguments it needs, and do what you need to do with 
ProcessBuilder - in a way irrespective of official docs - to make it work.

You got me playing with executables and Windows and Java some, so I am 
now stuck into it. :-)

AHS

[toc] | [prev] | [next] | [standalone]

Page 4 of 5 — ← Prev page 1 2 3 [4] 5  Next page →

Back to top | Article view | comp.lang.java.programmer

csiph-web