Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #94894 > unrolled thread

Most Pythonic way to store (small) configuration

Back to article view | Back to comp.lang.python

Contents

  Most Pythonic way to store (small) configuration Cecil Westerhof <Cecil@decebal.nl> - 2015-08-02 12:11 +0200
    Re: Most Pythonic way to store (small) configuration Chris Angelico <rosuav@gmail.com> - 2015-08-02 20:49 +1000
    Re: Most Pythonic way to store (small) configuration Ben Finney <ben+python@benfinney.id.au> - 2015-08-02 21:54 +1000
      Re: Most Pythonic way to store (small) configuration Cecil Westerhof <Cecil@decebal.nl> - 2015-08-02 18:51 +0200
        Re: Most Pythonic way to store (small) configuration Lele Gaifax <lele@metapensiero.it> - 2015-08-02 22:02 +0200
        Re: Most Pythonic way to store (small) configuration Cameron Simpson <cs@zip.com.au> - 2015-08-03 08:49 +1000
        Re: Most Pythonic way to store (small) configuration Ben Finney <ben+python@benfinney.id.au> - 2015-08-03 11:16 +1000
    Re: Most Pythonic way to store (small) configuration Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-08-02 16:12 +0100
    Re: Most Pythonic way to store (small) configuration Tim Chase <python.list@tim.thechases.com> - 2015-08-02 16:11 -0500
      Re: Most Pythonic way to store (small) configuration Dan Sommers <dan@tombstonezero.net> - 2015-08-03 04:02 +0000
        Re: Most Pythonic way to store (small) configuration Steven D'Aprano <steve@pearwood.info> - 2015-08-03 23:38 +1000
          Re: Most Pythonic way to store (small) configuration Chris Angelico <rosuav@gmail.com> - 2015-08-03 23:46 +1000
          Re: Most Pythonic way to store (small) configuration Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-08-03 15:37 +0100
    Re: Most Pythonic way to store (small) configuration marco.nawijn@colosso.nl - 2015-08-04 07:53 -0700
      Re: Most Pythonic way to store (small) configuration Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2015-08-04 19:06 +0200
        Re: Most Pythonic way to store (small) configuration marco.nawijn@colosso.nl - 2015-08-04 11:37 -0700
      Re: Most Pythonic way to store (small) configuration Ben Finney <ben+python@benfinney.id.au> - 2015-08-05 05:59 +1000
        Re: Most Pythonic way to store (small) configuration Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-08-05 18:32 +1000
          Re: Most Pythonic way to store (small) configuration Chris Angelico <rosuav@gmail.com> - 2015-08-05 20:01 +1000
      Re: Most Pythonic way to store (small) configuration Michael Torrie <torriem@gmail.com> - 2015-08-04 19:32 -0600
        Re: Most Pythonic way to store (small) configuration Grant Edwards <invalid@invalid.invalid> - 2015-08-05 14:00 +0000
      Re: Most Pythonic way to store (small) configuration random832@fastmail.us - 2015-08-04 22:44 -0400
      Re: Most Pythonic way to store (small) configuration Michael Torrie <torriem@gmail.com> - 2015-08-04 22:48 -0600
        Re: Most Pythonic way to store (small) configuration Rustom Mody <rustompmody@gmail.com> - 2015-08-04 21:55 -0700
          Re: Most Pythonic way to store (small) configuration Lele Gaifax <lele@metapensiero.it> - 2015-08-05 08:54 +0200
    Re: Most Pythonic way to store (small) configuration Tim Chase <python.list@tim.thechases.com> - 2015-08-05 08:18 -0500
      Re: Most Pythonic way to store (small) configuration Rustom Mody <rustompmody@gmail.com> - 2015-08-05 06:37 -0700
        Re: Most Pythonic way to store (small) configuration Tim Chase <python.list@tim.thechases.com> - 2015-08-05 15:55 -0500
          Re: Most Pythonic way to store (small) configuration Marko Rauhamaa <marko@pacujo.net> - 2015-08-06 00:47 +0300
            Re: Most Pythonic way to store (small) configuration Tim Chase <python.list@tim.thechases.com> - 2015-08-05 18:43 -0500
            Re: Most Pythonic way to store (small) configuration Chris Angelico <rosuav@gmail.com> - 2015-08-06 10:07 +1000
              Re: Most Pythonic way to store (small) configuration Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-08-06 17:33 +1000
                Re: Most Pythonic way to store (small) configuration Chris Angelico <rosuav@gmail.com> - 2015-08-06 17:51 +1000
          Re: Most Pythonic way to store (small) configuration Rustom Mody <rustompmody@gmail.com> - 2015-08-05 18:01 -0700
            Re: Most Pythonic way to store (small) configuration Rustom Mody <rustompmody@gmail.com> - 2015-08-05 18:06 -0700
    Re: Most Pythonic way to store (small) configuration Rustom Mody <rustompmody@gmail.com> - 2015-08-05 06:46 -0700
      Re: Most Pythonic way to store (small) configuration Steven D'Aprano <steve@pearwood.info> - 2015-08-06 00:08 +1000
        Re: Most Pythonic way to store (small) configuration Rustom Mody <rustompmody@gmail.com> - 2015-08-05 07:25 -0700

Page 2 of 2 — ← Prev page 1 [2]

#95018

On 2015-08-05, Michael Torrie <torriem@gmail.com> wrote:
> On 08/04/2015 01:59 PM, Ben Finney wrote:
>> marco.nawijn@colosso.nl writes:
>> 
>>> Why not use Python files itself as configuration files?
>> 
>> Because configuration data will be user-editable. (If it's not
>> user-editable, that is itself a poor design choice.)
>> 
>> If you allow executable code to be user-edited, that opens your program
>> to arbitrary injection of executable code. Your program becomes wide
>> open for security exploits, whether through malicious or accidental
>> bugs, and simple human error can lead to arbitrary-scope damage to the
>> user's system.
>
> We need to state the context here.  The only context in which having a
> Python config file is dangerous is when the python program runs as a
> different user/privilege than the owner of the config file.  If the user
> owns the python files as well as the config file then none of this matters.

Yes, it does.

We're not just talking about intentional, malicious damange, we're
also talking about _accidental_ damage caused by an incorrect edit of
a configuration files.

It's much harder to cause damage by mis-editing an "ini" format file
that's parsed with the config file library than it is by mis-editing a
Python file that's imported.

-- 
Grant Edwards               grant.b.edwards        Yow! Clear the laundromat!!
                                  at               This whirl-o-matic just had
                              gmail.com            a nuclear meltdown!!

[toc] | [prev] | [next] | [standalone]

#94989

On Tue, Aug 4, 2015, at 21:32, Michael Torrie wrote:
> In many of my projects I put basic config variables in a file like
> config.py and import that in each module that needs it.  The config
> module doubles as a global namespace for sharing between modules as well.

What about JSONP? That is, a file consisting exactly of "config_data =
[JSON object]" That would get you some of the benefits of having your
config file exist as a python module, but still allow it to be examined
by other tools, written out, etc.

[toc] | [prev] | [next] | [standalone]

#94991

On 08/04/2015 08:44 PM, random832@fastmail.us wrote:
> On Tue, Aug 4, 2015, at 21:32, Michael Torrie wrote:
>> In many of my projects I put basic config variables in a file like
>> config.py and import that in each module that needs it.  The config
>> module doubles as a global namespace for sharing between modules as well.
> 
> What about JSONP? That is, a file consisting exactly of "config_data =
> [JSON object]" That would get you some of the benefits of having your
> config file exist as a python module, but still allow it to be examined
> by other tools, written out, etc.

But I don't need it to be examined by other tools.  So the added
complication of yet another layer isn't worth it or needed.  Python's
syntax is simple enough that a person with a text editor can certainly
do it.  Again, context is everything.  My programs are written for
mostly my own use.  If I was writing a system that was like, say,
Apache, I would certainly do a DSL with a robust error checking and
reporting system that could clearly help people with syntax errors, etc.

[toc] | [prev] | [next] | [standalone]

#94992

On Wednesday, August 5, 2015 at 10:19:11 AM UTC+5:30, Michael Torrie wrote:
> On 08/04/2015 08:44 PM,  wrote:
> > On Tue, Aug 4, 2015, at 21:32, Michael Torrie wrote:
> >> In many of my projects I put basic config variables in a file like
> >> config.py and import that in each module that needs it.  The config
> >> module doubles as a global namespace for sharing between modules as well.
> > 
> > What about JSONP? That is, a file consisting exactly of "config_data =
> > [JSON object]" That would get you some of the benefits of having your
> > config file exist as a python module, but still allow it to be examined
> > by other tools, written out, etc.
> 
> But I don't need it to be examined by other tools.  So the added
> complication of yet another layer isn't worth it or needed.  Python's
> syntax is simple enough that a person with a text editor can certainly
> do it.  Again, context is everything.  My programs are written for
> mostly my own use.  If I was writing a system that was like, say,
> Apache, I would certainly do a DSL with a robust error checking and
> reporting system that could clearly help people with syntax errors, etc.

I think the main point is Cameron's

> So on the whole I am against python code as the config file format. Really,
> who needs a Turing complete configuration file?

stated more strongly: I sure dont want to use something Turing complete
from something that is inherently more trivial.
And if the cost is of matching the trivial format to the trivial-izing reader
(say json) it seems like a small price to pay.
In the past Ive always recommended yaml even though the cost is higher --
a separate install [yeah Ive had clients tell me that yaml's out because of that]
[Does yaml have comments?]

[toc] | [prev] | [next] | [standalone]

#94996

Rustom Mody <rustompmody@gmail.com> writes:

> Does yaml have comments?

Yes, the same syntax as Python's.

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

[toc] | [prev] | [next] | [standalone]

#95015

On 2015-08-02 12:11, Cecil Westerhof wrote:
> There are a lot of ways to store configuration information:
> - conf file
> - xml file
> - database
> - json file
> - and possible a lot of other ways
> 
> I want to write a Python program to display cleaned log files. I do
> not think I need a lot of configuration to be stored:
> - some things relating to the GUI
> - default behaviour
> - default directory
> - log files to display, including some info
>   - At least until where it was displayed
> 
> Because of this I think a human readable file would be best.

Yet another mostly-built-in option is to just have a simple file of
key/value pairs, optionally with comments.  This can be read with
something like

  config = {}
  with open('config.ini') as f:
    for row in f:
      row = row.strip()
      if not row or row.startswith(('#', ';')):
        continue
      k, _, v = row.partition('=')
      config[k.strip().upper()] = v.lstrip()

which is pretty straight-forward and easy format to edit.

-tkc

[toc] | [prev] | [next] | [standalone]

#95016

On Wednesday, August 5, 2015 at 6:58:01 PM UTC+5:30, Tim Chase wrote:
> On 2015-08-02 12:11, Cecil Westerhof wrote:
> > There are a lot of ways to store configuration information:
> > - conf file
> > - xml file
> > - database
> > - json file
> > - and possible a lot of other ways
> > 
> > I want to write a Python program to display cleaned log files. I do
> > not think I need a lot of configuration to be stored:
> > - some things relating to the GUI
> > - default behaviour
> > - default directory
> > - log files to display, including some info
> >   - At least until where it was displayed
> > 
> > Because of this I think a human readable file would be best.
> 
> Yet another mostly-built-in option is to just have a simple file of
> key/value pairs, optionally with comments.  This can be read with
> something like
> 
>   config = {}
>   with open('config.ini') as f:
>     for row in f:
>       row = row.strip()
>       if not row or row.startswith(('#', ';')):
>         continue
>       k, _, v = row.partition('=')
>       config[k.strip().upper()] = v.lstrip()
> 
> which is pretty straight-forward and easy format to edit.
> 
> -tkc

JSON handles basic types like this:
>>> from json import loads
>>> loads("""{"anInt":1, "aString":"2"}""")
{'aString': '2', 'anInt': 1}

[toc] | [prev] | [next] | [standalone]

#95036

On 2015-08-05 06:37, Rustom Mody wrote:
> >   config = {}
> >   with open('config.ini') as f:
> >     for row in f:
> >       row = row.strip()
> >       if not row or row.startswith(('#', ';')):
> >         continue
> >       k, _, v = row.partition('=')
> >       config[k.strip().upper()] = v.lstrip()
> > 
> > which is pretty straight-forward and easy format to edit.
> > 
> > -tkc  
> 
> JSON handles basic types like this:
> >>> from json import loads
> >>> loads("""{"anInt":1, "aString":"2"}""")  
> {'aString': '2', 'anInt': 1}

But requires the person hand-editing the file to make sure that
opening braces close, that quoted text is properly opened/closed, has
peculiarities regarding things following back-slashes, etc.

There's a certain simplicity to simply having key/value pairs
separated by an "=" and then letting the application do whatever it
needs/wants with those key/value strings.

-tkc

[toc] | [prev] | [next] | [standalone]

#95039

Tim Chase <python.list@tim.thechases.com>:

> There's a certain simplicity to simply having key/value pairs
> separated by an "=" and then letting the application do whatever it
> needs/wants with those key/value strings.

That trap has lured in a lot of wildlife.

What to do with lists?

Is whitespace significant?

Case in point, systemd configuration files:

   <URL: http://www.freedesktop.org/software/systemd/man/systemd.servic
   e.html#Command%20lines>

It specifies all kinds of application-do-whatever-it-needs syntax:

 * backslash escapes

 * double quotes and single quotes

 * line continuations

 * percent specifiers

 * dollar substitution with or without braces

 * double-dollar escape

 * together with undocumented quirks (for example, how do you specify a
   command whose pathname contains whitespace?)

All of which makes it all but impossible to algorithmically escape a
literal command into the ExecStart= entry.

When you start with something that's too simple, you end up with layers
of ever-increasing complexity but never attain the expressive power of
JSON, S-expressions and the like (I don't have the stomach to mention
XML).


Marko

[toc] | [prev] | [next] | [standalone]

#95043

On 2015-08-06 00:47, Marko Rauhamaa wrote:
> > There's a certain simplicity to simply having key/value pairs
> > separated by an "=" and then letting the application do whatever
> > it needs/wants with those key/value strings.  
> 
> That trap has lured in a lot of wildlife.
> 
> What to do with lists?
> 
> Is whitespace significant?

As I said, simple.  Lists?  Not in a simple config format.  Though I
suppose, just like the .ini format, you could use a convention of
comma or space-separated list items. Or quoted white-space-separated
items like shlex handles nicely.  Or if you really need, pass the
value portion of the key/value pair through ast.literal_eval()

Significant whitespace?  Not usually simple (just stuck touching a
project where someone committed with tons of trailing whitespaces.
grumble), so strip 'em off as if they're an error condition.  I've
never had a config-file where I wanted leading/trailing whitespace as
significant.

> Case in point, systemd configuration files:

The antithesis of "simplicity" ;-)

-tkc

[toc] | [prev] | [next] | [standalone]

#95044

On Thu, Aug 6, 2015 at 9:43 AM, Tim Chase <python.list@tim.thechases.com> wrote:
> Significant whitespace?  Not usually simple (just stuck touching a
> project where someone committed with tons of trailing whitespaces.
> grumble), so strip 'em off as if they're an error condition.  I've
> never had a config-file where I wanted leading/trailing whitespace as
> significant.

If you're configuring a prompt, sometimes you need to be able to
include a space at the end of it. Since trailing whitespace on a line
in the file itself is a bad idea, you need some way of marking it.
That might mean quoting the string, or having a Unicode or byte escape
like \x20 that means space, or something like that. If you define that
spaces around your equals sign are insignificant, you need the same
sort of system to cope with the possibility of actual leading
whitespace, too.

>> Case in point, systemd configuration files:
>
> The antithesis of "simplicity" ;-)

Ehh... I reckon they're pretty simple. They're somewhat more powerful
than Upstart config files, and pay some complexity cost for that; but
they're a lot simpler than sysvinit "config files" (which are actually
shell scripts), especially with all the dependency handling cruft that
goes into encoded comments. Frankly, I do my best to avoid ever
touching those. I'm not sure what else to compare them against (never
used any init system other than the three just mentioned), so I don't
see systemd files as being particularly complex.

ChrisA

[toc] | [prev] | [next] | [standalone]

#95074

On Thursday 06 August 2015 10:07, Chris Angelico wrote:

> On Thu, Aug 6, 2015 at 9:43 AM, Tim Chase <python.list@tim.thechases.com>
> wrote:
>> Significant whitespace?  Not usually simple (just stuck touching a
>> project where someone committed with tons of trailing whitespaces.
>> grumble), so strip 'em off as if they're an error condition.  I've
>> never had a config-file where I wanted leading/trailing whitespace as
>> significant.
> 
> If you're configuring a prompt, sometimes you need to be able to
> include a space at the end of it.

"Sometimes"? What sort of filthy perv doesn't separate their prompts from 
the user input with at least one space? Disgusting, I call it.

    this is a prompt and this isn't

versus

    this is a promptand this isn't

Come the revolution, anyone who writes the second will be taken out and 
shot.

The right solution to that is to have the display function add a space to 
the end of the prompt if there isn't already one, that way you don't need to 
care about putting a space at the end of the prompt yourself.


> Since trailing whitespace on a line
> in the file itself is a bad idea, you need some way of marking it.

I'm partial to one of two schemes:

- strings need to be quoted, so leading and trailing spaces are easy: 

  key = " this has both leading and trailing spaces "

- strings don't need to be quoted, and spaces are significant *except* 
  at  the ends of the string:

  key = this has no leading or trailing spaces

  If you need spaces at the end, use an escape sequence:

  key = \sthis has both leading and trailing spaces\s

say, or ^SP or \N{SPACE} or whatever floats your boat. Unless your 
requirements are limited to only printable ASCII strings, you're going to 
need to provide some sort of escape sequence anyway, to allow (say) 
newlines.


-- 
Steve

[toc] | [prev] | [next] | [standalone]

#95075

On Thu, Aug 6, 2015 at 5:33 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Thursday 06 August 2015 10:07, Chris Angelico wrote:
>
>> On Thu, Aug 6, 2015 at 9:43 AM, Tim Chase <python.list@tim.thechases.com>
>> wrote:
>>> Significant whitespace?  Not usually simple (just stuck touching a
>>> project where someone committed with tons of trailing whitespaces.
>>> grumble), so strip 'em off as if they're an error condition.  I've
>>> never had a config-file where I wanted leading/trailing whitespace as
>>> significant.
>>
>> If you're configuring a prompt, sometimes you need to be able to
>> include a space at the end of it.
>
> "Sometimes"? What sort of filthy perv doesn't separate their prompts from
> the user input with at least one space? Disgusting, I call it.
>
>     this is a prompt and this isn't
>
> versus
>
>     this is a promptand this isn't
>
> Come the revolution, anyone who writes the second will be taken out and
> shot.

Oh, well, there are these people in backwards nations that still have
prompts that look like this:

C:\>

and have no space after them. I *was* trying to be courteous, but
maybe courtesy is wasted on those who'll be shot come the revolution
anyway.

> The right solution to that is to have the display function add a space to
> the end of the prompt if there isn't already one, that way you don't need to
> care about putting a space at the end of the prompt yourself.

That's an option that solves one specific instance of the problem, but
I'm sure there are other places where you may or may not want a
trailing space, so it needs a marker.

>> Since trailing whitespace on a line
>> in the file itself is a bad idea, you need some way of marking it.
>
> I'm partial to one of two schemes:
>
> - strings need to be quoted, so leading and trailing spaces are easy:
>
>   key = " this has both leading and trailing spaces "

Yeah, this is fine for a lot of cases. It does add small complexity,
though, to the simple case where you're setting simple tokens as
values.

> - strings don't need to be quoted, and spaces are significant *except*
>   at  the ends of the string:
>
>   key = this has no leading or trailing spaces
>
>   If you need spaces at the end, use an escape sequence:
>
>   key = \sthis has both leading and trailing spaces\s
>
> say, or ^SP or \N{SPACE} or whatever floats your boat. Unless your
> requirements are limited to only printable ASCII strings, you're going to
> need to provide some sort of escape sequence anyway, to allow (say)
> newlines.

This is starting to get toward a full-blown DSL, which is costly. At
this point, you may as well go for something well-known, which is what
this thread's all about.

A compromise might be: Unquoted strings get trimmed and are restricted
to one line, but if you want leading/trailing spaces, or
leading/trailing quote characters, or embedded newlines, or Unicode
escapes, you must first put quotes around the string.

server = 127.0.0.1
message = "Hi there!\nTwo line message being delivered."
windows_path = c:\users\adam

That way, you pay the complexity cost of full string parsing only if
you actually need it.

ChrisA

[toc] | [prev] | [next] | [standalone]

#95048

On Thursday, August 6, 2015 at 2:31:52 AM UTC+5:30, Tim Chase wrote:
> On 2015-08-05 06:37, Rustom Mody wrote:
> > >   config = {}
> > >   with open('config.ini') as f:
> > >     for row in f:
> > >       row = row.strip()
> > >       if not row or row.startswith(('#', ';')):
> > >         continue
> > >       k, _, v = row.partition('=')
> > >       config[k.strip().upper()] = v.lstrip()
> > > 
> > > which is pretty straight-forward and easy format to edit.
> > > 
> > > -tkc  
> > 
> > JSON handles basic types like this:
> > >>> from json import loads
> > >>> loads("""{"anInt":1, "aString":"2"}""")  
> > {'aString': '2', 'anInt': 1}
> 
> But requires the person hand-editing the file to make sure that
> opening braces close, that quoted text is properly opened/closed, has
> peculiarities regarding things following back-slashes, etc.
> 
> There's a certain simplicity to simply having key/value pairs
> separated by an "=" and then letting the application do whatever it
> needs/wants with those key/value strings.
> 
> -tkc

I just checked that literal_eval accepts comments.
So thats one plus for that.
However I must admit that in checking that out I was faced with more than
(I) expected unfriendly error messages like

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/ast.py", line 84, in literal_eval
    return _convert(node_or_string)
  File "/usr/lib/python3.4/ast.py", line 62, in _convert
    in zip(node.keys, node.values))
  File "/usr/lib/python3.4/ast.py", line 61, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/usr/lib/python3.4/ast.py", line 83, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x7fe1173ebac8>


By contrast here is a more friendly error message (had put a comma where a colon
required)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/ast.py", line 46, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/lib/python3.4/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 2
    "i", 1}

So overall whether ast.literal_eval is a good idea is not clear to me

[toc] | [prev] | [next] | [standalone]

#95050

On Thursday, August 6, 2015 at 6:32:03 AM UTC+5:30, Rustom Mody wrote:

> By contrast here is a more friendly error message (had put a comma where a colon
> required)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python3.4/ast.py", line 46, in literal_eval
>     node_or_string = parse(node_or_string, mode='eval')
>   File "/usr/lib/python3.4/ast.py", line 35, in parse
>     return compile(source, filename, mode, PyCF_ONLY_AST)
>   File "<unknown>", line 2
>     "i", 1}

Uh...
The more helpfulness not evident as the most crucial line not cut-pasted

Heres the full bt

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/ast.py", line 46, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/lib/python3.4/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 2
    "i", 1}
       ^
SyntaxError: invalid syntax

[toc] | [prev] | [next] | [standalone]

#95017

On Sunday, August 2, 2015 at 3:44:51 PM UTC+5:30, Cecil Westerhof wrote:
> There are a lot of ways to store configuration information:
> - conf file
> - xml file
> - database
> - json file
> - and possible a lot of other ways

One that I dont think has been mentioned:
ast.literal_eval

[toc] | [prev] | [next] | [standalone]

#95019

On Wed, 5 Aug 2015 11:46 pm, Rustom Mody wrote:

> On Sunday, August 2, 2015 at 3:44:51 PM UTC+5:30, Cecil Westerhof wrote:
>> There are a lot of ways to store configuration information:
>> - conf file
>> - xml file
>> - database
>> - json file
>> - and possible a lot of other ways
> 
> One that I dont think has been mentioned:
> ast.literal_eval


Probably because it doesn't work :-)

py> import ast
py> s = "x = 23"  # pretend I read this line from a file
py> ast.literal_eval(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/ast.py", line 47, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/local/lib/python3.3/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    x = 23
      ^
SyntaxError: invalid syntax


You might be able to build Yet Another Config Format using literal_eval as a
building block, for evaluating values, but *in and of itself* it isn't a
way to store config information, any more than

int
float

or any other function which takes a string and evaluates it as a Python
primitive or built-in type.


However, there are at least config formats in the standard library which I
believe we've missed: shelve, and plistlib.

help(shelve)

    A "shelf" is a persistent, dictionary-like object.  The difference
    with dbm databases is that the values (not the keys!) in a shelf can
    be essentially arbitrary Python objects -- anything that the "pickle"
    module can handle.  This includes most class instances, recursive data
    types, and objects containing lots of shared sub-objects.  The keys
    are ordinary strings.


help(plistlib)

    The property list (.plist) file format is a simple XML pickle supporting
    basic object types, like dictionaries, lists, numbers and strings.
    Usually the top level object is a dictionary.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#95020

On Wednesday, August 5, 2015 at 7:38:46 PM UTC+5:30, Steven D'Aprano wrote:
> On Wed, 5 Aug 2015 11:46 pm, Rustom Mody wrote:
> 
> > On Sunday, August 2, 2015 at 3:44:51 PM UTC+5:30, Cecil Westerhof wrote:
> >> There are a lot of ways to store configuration information:
> >> - conf file
> >> - xml file
> >> - database
> >> - json file
> >> - and possible a lot of other ways
> > 
> > One that I dont think has been mentioned:
> > ast.literal_eval
> 
> 
> Probably because it doesn't work :-)

In the way you describe... yes
Works alright if you give it *literals*

>>> from ast import literal_eval
>>> literal_eval('{"x":"hello", "y":2, "z":3.142}')
{'z': 3.142, 'x': 'hello', 'y': 2}

which is identical the the json.loads behavior
>>> loads('{"x":"hello", "y":2, "z":3.142}')
{'z': 3.142, 'x': 'hello', 'y': 2}

of course the natural method of use would of course get the string from
the open-ing of a config file -- something along the lines

configuration = literal_eval(open("~/.fooconfig"))

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web