Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #6294 > unrolled thread

Parse config file and command-line arguments, to get a single collection of options

Back to article view | Back to comp.lang.python

Contents

  Parse config file and command-line arguments, to get a single collection of options Ben Finney <ben@benfinney.id.au> - 2011-05-26 14:38 +1000
    Re: Parse config file and command-line arguments, to get a single collection of options Raymond Hettinger <python@rcn.com> - 2011-05-25 22:15 -0700
    Re: Parse config file and command-line arguments, to get a single collection of options rzed <rzantow@gmail.com> - 2011-05-28 23:12 +0000

#6294 — Parse config file and command-line arguments, to get a single collection of options

Howdy all,

Python's standard library has modules for configuration file parsing
(configparser) and command-line argument parsing (optparse, argparse). I
want to write a program that does both, but also:

* Has a cascade of options: default option values, overridden by config
  file options, overridden by command-line options.

* Reads a different, or even additional, configuration file if specified
  on the command-line (e.g. --config-file foo.conf) and yet still obeys
  the above cascade.

* Allows a single definition of an option (e.g. logging level) to define
  the same option for parsing from configuration files and the command
  line.

* Unifies the parsed options into a single collection for the rest of
  the program to access without caring where they came from.

How can I achieve this with minimum deviation from the Python standard
library?


(For anyone interested in gaining StackOverflow points, I'm also asking
this as a question there so feel free to post answers on that site
<URL:http://stackoverflow.com/questions/6133517/parse-config-file-and-command-line-arguments-to-get-a-single-collection-of-optio>.)

-- 
 \         “Apologize, v. To lay the foundation for a future offense.” |
  `\                   —Ambrose Bierce, _The Devil's Dictionary_, 1906 |
_o__)                                                                  |
Ben Finney

[toc] | [next] | [standalone]

#6299

On May 25, 9:38 pm, Ben Finney <b...@benfinney.id.au> wrote:
> Howdy all,
>
> Python's standard library has modules for configuration file parsing
> (configparser) and command-line argument parsing (optparse, argparse). I
> want to write a program that does both, but also:
>
> * Has a cascade of options: default option values, overridden by config
>   file options, overridden by command-line options.
>
> * Reads a different, or even additional, configuration file if specified
>   on the command-line (e.g. --config-file foo.conf) and yet still obeys
>   the above cascade.
>
> * Allows a single definition of an option (e.g. logging level) to define
>   the same option for parsing from configuration files and the command
>   line.
>
> * Unifies the parsed options into a single collection for the rest of
>   the program to access without caring where they came from.
>
> How can I achieve this with minimum deviation from the Python standard
> library?

One thought is start with something like ChainMap,
http://code.activestate.com/recipes/305268-chained-map-lookups/?in=user-178123
, or some variant to unify multiple mapping objects into a single
prioritized collection.  A mapping for command line args can be made
by using vars() on an argparse namespace to create a dictionary.
ConfigParser's mapping is accessible via its get() method.  With a
ChainMap style object you can add other option sources such as
os.environ.  This should get you started on your grand unified, do-
everything-at-once vision with minimal deviation from the standard
library.

Raymond

[toc] | [prev] | [next] | [standalone]

#6495

Ben Finney <ben@benfinney.id.au> wrote in
news:87k4deaxfc.fsf@benfinney.id.au: 

> Howdy all,
> 
> Python's standard library has modules for configuration file
> parsing (configparser) and command-line argument parsing
> (optparse, argparse). I want to write a program that does both,
> but also: 
> 
> * Has a cascade of options: default option values, overridden by
> config 
>   file options, overridden by command-line options.
> 
> * Reads a different, or even additional, configuration file if
> specified 
>   on the command-line (e.g. --config-file foo.conf) and yet still
>   obeys the above cascade.
> 
> * Allows a single definition of an option (e.g. logging level) to
> define 
>   the same option for parsing from configuration files and the
>   command line.
> 
> * Unifies the parsed options into a single collection for the
> rest of 
>   the program to access without caring where they came from.
> 
> How can I achieve this with minimum deviation from the Python
> standard library?
> 
> 
> (For anyone interested in gaining StackOverflow points, I'm also
> asking this as a question there so feel free to post answers on
> that site 
> <URL:http://stackoverflow.com/questions/6133517/parse-config-file-
> and-command-line-arguments-to-get-a-single-collection-of-optio>.) 
> 

This seems vaguely similar to a module I wrote and use all the time. 
It allows default value specification, categorization of command-line 
options, in-line parsing of further spec file[s] and overrides of 
values in the the sequence you define, pretty much. It doesn't deal 
with the "logging level" item. I'm not sure what that would mean. The  
idea of the module is to create a namespace object (though it could 
as easily be a dict) that contains whatever values are specified.

in the program it would be invoked like this:
ctx = Cmdline( outfname='test.out', size=(250,400), ... ) (or 
whatever).

The command line can contain switches, prefixed by hyphens, spec file 
names, prefixed by @, untagged names, or key=value pairs. The values 
will be parsed as (tuples), [lists], or {dicts}, ints, floats, or 
strings. I did not, I am ashamed to say, resist the temptation to 
guess. 'Ctx' will contain the result of all this. Switches, if any 
are in a list named ctx._switches, unadorned arguments are in a list 
named ctx._vars, and the other stuff is pretty much as you would 
expect. It expects configuration files to be in a sort of ini-file 
format. 

Here's an example:

---------------------------
test.spec:
log=test.log
count=10
size=(400,200)
group1={key=value,a=alpha}
group2={b=beta,name=doogie howser}
#
temp=[1,2,5,11,22]
sub=(al,becky,carl,diane,edwin,fran)

---------------------------
test.py:
from Cmdline import Cmdline
c = Cmdline( count=5, log='pink.tank')
c.show()

---------------------------
>python test.py log=friend.txt @test.spec count=32 name=waldo

... yields:
<type 'int'> count  = 32
<type 'tuple'> sub  = ('al', 'becky', 'carl', 'diane', 'edwin', 
'fran')
<type 'str'> log  = 'test.log'
<type 'list'> temp  = [1, 2, 5, 11, 22]
<type 'dict'> group1  = {'a': 'alpha', 'key': 'value'}
<type 'dict'> group2  = {'b': 'beta', 'name': 'doogie howser'}
<type 'tuple'> size  = (400, 200)
<type 'str'> name  = 'waldo'

... while

>python @test.spec count=32 name=waldo temp=Vitalis sub='' dream -k
<type 'int'> count  = 32
<type 'list'> _vars  = ['dream']
<type 'str'> log  = 'test.log'
<type 'str'> temp  = 'Vitalis'
<type 'list'> _switches  = ['k']
<type 'str'> name  = 'waldo'
<type 'dict'> group1  = {'a': 'alpha', 'key': 'value'}
<type 'dict'> group2  = {'b': 'beta', 'name': 'doogie howser'}
<type 'tuple'> size  = (400, 200)
<type 'str'> sub  = ''

What the program does with the results is up to it, of course.

-- 
rzed

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web