Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110533 > unrolled thread

argparse and subparsers

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  argparse and subparsers "Joseph L. Casale" <jcasale@activenetwerx.com> - 2016-06-26 18:51 +0000
    Re: argparse and subparsers Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-06-26 20:15 -0700
    Re: argparse and subparsers Sachin Garg <s.garg.computer@gmail.com> - 2016-06-27 00:55 -0400
      Re: argparse and subparsers Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-06-26 22:34 -0700
        Re: argparse and subparsers Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-06-27 20:28 +1000
          Re: argparse and subparsers Sachin Garg <s.garg.computer@gmail.com> - 2016-06-27 13:22 -0400
            Re: argparse and subparsers Michele Simionato <michele.simionato@gmail.com> - 2016-06-28 03:02 -0700
      RE: argparse and subparsers "Joseph L. Casale" <jcasale@activenetwerx.com> - 2016-06-28 02:22 +0000

#110533 — argparse and subparsers

I have some code where sys.argv is sliced up and manually fed to discrete argparse
instances each with a single subparser. The reason the discrete parsers all having a
single subparser was to make handling the input simpler, the first arg in the slice
could be left in.

This has become unmaintainable as the manual slicing is always subject to a new case
by where a parser has a positional, switch or optional parameter for example. Also, since
argv is grouped by subparser specifiers, if a parameter has input that matches a keyword
it all goes pear shaped.

The underlying root of this mess is a long unaddressed limitation in argparse to support
multiple subparser specifications on the same invocation:

prog.py -x -y 42 -foo bar subParserA -a 1 -b 2 subParserB -a 1 -b 2 subParserB -a 1 -b 2

The base arguments (-x -y 42 -foo bar).
An invocation of "subParserA" and its arguments (-a 1 -b 2).
Two invocations of "subParserB" and their arguments.

etc...

I have seen some creative ways to overcome this on stacktrace, however I thought I'd
see what people here have done. The code is pinned at 2.7 and while several people
have created alternate implementations which address many of argparses failures, its
desired to stick to base lib but that can easily be changed given a compelling reason
if an alternate implementation exists that works well.

Thanks for any thoughts,
jlc

[toc] | [next] | [standalone]

#110553

On Monday, June 27, 2016 at 6:56:41 AM UTC+12, Joseph L. Casale wrote:
> This has become unmaintainable as the manual slicing is always subject
> to a new case by where a parser has a positional, switch or optional
> parameter for example. Also, since argv is grouped by subparser
> specifiers, if a parameter has input that matches a keyword
> it all goes pear shaped.

Write your own parser. It’s not hard--the common Unix/Linux command line conventions are quite simple, and it’s already pre-split into words for you.

[toc] | [prev] | [next] | [standalone]

#110560

On Sunday 26 June 2016 02:51 PM, Joseph L. Casale wrote:
> I have some code where sys.argv is sliced up and manually fed to discrete argparse
> instances each with a single subparser. The reason the discrete parsers all having a
> single subparser was to make handling the input simpler, the first arg in the slice
> could be left in.
> 
> This has become unmaintainable as the manual slicing is always subject to a new case
> by where a parser has a positional, switch or optional parameter for example. Also, since
> argv is grouped by subparser specifiers, if a parameter has input that matches a keyword
> it all goes pear shaped.
> 
> The underlying root of this mess is a long unaddressed limitation in argparse to support
> multiple subparser specifications on the same invocation:
> 
> prog.py -x -y 42 -foo bar subParserA -a 1 -b 2 subParserB -a 1 -b 2 subParserB -a 1 -b 2
> 
> The base arguments (-x -y 42 -foo bar).
> An invocation of "subParserA" and its arguments (-a 1 -b 2).
> Two invocations of "subParserB" and their arguments.
> 
> etc...
> 
> I have seen some creative ways to overcome this on stacktrace, however I thought I'd
> see what people here have done. The code is pinned at 2.7 and while several people
> have created alternate implementations which address many of argparses failures, its
> desired to stick to base lib but that can easily be changed given a compelling reason
> if an alternate implementation exists that works well.


Not sure if this fits the bill, or makes sense here, but I came cross
"docopt" which touts itself as a "Command-line interface description
language". I used it in a project and it seems to be pretty easy to use
as well as elegant. It stores the arguments & values as a dictionary,
keyed by the argument.

from docopt import docopt

arguments = docopt(__doc__, version='0.2')

# Set verbose flag
verbose = False
if arguments['--verbose']:
    verbose = True
elif arguments['-q']:
    verbose = False

# If --noencrypt, --nosign or --notransfer is specified, put that in config
if arguments['--no-encrypt']:
    config['noencrypt'] = True
else:
    config['noencrypt'] = False

if arguments['--no-sign']:
    config['nosign'] = True
else:
    config['nosign'] = False

if arguments['--no-transfer']:
    config['notransfer'] = True
else:
    config['notransfer'] = False

and so on ...

[toc] | [prev] | [next] | [standalone]

#110561

On Monday, June 27, 2016 at 4:56:10 PM UTC+12, Sachin Garg wrote:

> # Set verbose flag
> verbose = False
> if arguments['--verbose']:
>     verbose = True
> elif arguments['-q']:
>     verbose = False

Don’t you just love code (and commenting) like this...

[toc] | [prev] | [next] | [standalone]

#110570

On Monday 27 June 2016 15:34, Lawrence D’Oliveiro wrote:

> On Monday, June 27, 2016 at 4:56:10 PM UTC+12, Sachin Garg wrote:
> 
>> # Set verbose flag
>> verbose = False
>> if arguments['--verbose']:
>>     verbose = True
>> elif arguments['-q']:
>>     verbose = False
> 
> Don’t you just love code (and commenting) like this...


Not particularly, but what are you going to do? You need to support a minimum 
of three cases:

- a default setting;
- the case where the user passes --verbose;
- the case where the user passes -q;

(I'm not sure why --verbose take a long argument but no short argument -v; and 
likewise why there's a -q short argument but no --quiet long version. Oh well.)

A simple test-and-set for each argument is the simplest, most straight-forward 
way of handling this. It's not *pretty* or *elegant* code, but it is the 
easiest to read, write and comprehend, and in my opinion much better than 
alternatives involving ever more arcane method calls to ever more complicated 
classes.

I don't have experience with docutils and cannot judge whether or not Sachin's 
snippets are good or bad examples of use, but find myself going back to the 
good old fashioned GNU style command line parser whenever I need a few command 
line options. If you find yourself writing subparsers and "mandatory options" 
and needing entire help screens to describe single arguments (as in "foo --help 
arg") then really I think you should give up the pretence that you're dealing 
with command line options, and you should write a mini-language for your 
application.

(hg, git, memcoder, soc etc. I'm talking about you.)



-- 
Steve

[toc] | [prev] | [next] | [standalone]

#110602

On Monday 27 June 2016 06:28 AM, Steven D'Aprano wrote:
> On Monday 27 June 2016 15:34, Lawrence D’Oliveiro wrote:
> 
>> On Monday, June 27, 2016 at 4:56:10 PM UTC+12, Sachin Garg wrote:
>>
>>> # Set verbose flag
>>> verbose = False
>>> if arguments['--verbose']:
>>>     verbose = True
>>> elif arguments['-q']:
>>>     verbose = False
>>
>> Don’t you just love code (and commenting) like this...
> 
> 
> Not particularly, but what are you going to do? You need to support a minimum 
> of three cases:
> 
> - a default setting;
> - the case where the user passes --verbose;
> - the case where the user passes -q;
> 
> (I'm not sure why --verbose take a long argument but no short argument -v; and 
> likewise why there's a -q short argument but no --quiet long version. Oh well.)

It is there. The way docopt works (https://github.com/docopt/docopt) is
that it uses a "usage pattern" formatted using doctring conventions. In
my case (snippet below), this pattern was:

"""Process Files: De-identify, encrypt and transmit data.

Usage:
    processFiles.py [-hvq] [--config=FILE] [--encryptkey=key]
[--signkey=key] [--no-normalize] [--no-encrypt] [--no-sign]

Options:
    -h --help           show this help message and exit
    -v --verbose        verbose mode
    -q                  quiet mode (default)

    --config=FILE       read configuration FILE (default: config.json)

    --encryptkey=KEY    set GPG encryption key to KEY(default: from
config file)
    --signkey=KEY       set GPG signing key to KEY (default: from config
file)
    --no-normalize      do not normalize CCD/CSV (default: normalize)
    --no-encrypt        do not encrypt output (default: encrypt)
    --no-sign           do not sign output (default: sign)
    --no-transfer       do not transfer output (default: transfer)
"""

So, the short "-v" option is taken care  of.

> A simple test-and-set for each argument is the simplest, most straight-forward 
> way of handling this. It's not *pretty* or *elegant* code, but it is the 
> easiest to read, write and comprehend, and in my opinion much better than 
> alternatives involving ever more arcane method calls to ever more complicated 
> classes.

The code above does seem amateurish. However, I think that it is easier
to "waste" a few variables and allow for the ability to do printf()
debugging, then write code using esoteric data structures.

> I don't have experience with docutils and cannot judge whether or not Sachin's 
> snippets are good or bad examples of use, but find myself going back to the 
> good old fashioned GNU style command line parser whenever I need a few command 
> line options. If you find yourself writing subparsers and "mandatory options" 
> and needing entire help screens to describe single arguments (as in "foo --help 
> arg") then really I think you should give up the pretence that you're dealing 
> with command line options, and you should write a mini-language for your 
> application.
> 
> (hg, git, memcoder, soc etc. I'm talking about you.)

[toc] | [prev] | [next] | [standalone]

#110674

I did not know about docopt. It is basically the same idea of this recipe I wrote about 12 years ago:

https://code.activestate.com/recipes/278844-parsing-the-command-line/?in=user-1122360

Good that it was reinvented :-)

[toc] | [prev] | [next] | [standalone]

#110634

> Not sure if this fits the bill, or makes sense here, but I came cross
> "docopt" which touts itself as a "Command-line interface description
> language". I used it in a project and it seems to be pretty easy to use
> as well as elegant. It stores the arguments & values as a dictionary,
> keyed by the argument.

Yea I have had my eye on docopt for a while, it doesn't support multiple
subparsers, as I am passing duplicate parameters from its perspective.

Imagine:

foo.py --host 172.18.0.4 --port 766 foo --warning 42 --critical 77 bar --warning 4.2 --critical 7.7

etc...

The shortcoming to argparse has been debated to death and the bug tracker
was just left hanging. To be honest, I am not clear on the opposition to it...

Thanks,
jlc

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web