Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #10611 > unrolled thread

Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong?

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong? Andrew Berg <bahamutzero8825@gmail.com> - 2011-07-30 22:10 -0500
    Re: Deeply nested dictionaries - should I look into a database or   am I just doing it wrong? Thorsten Kampe <thorsten@thorstenkampe.de> - 2011-07-31 09:41 +0200
      Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong? Andrew Berg <bahamutzero8825@gmail.com> - 2011-07-31 13:36 -0500
        Re: Deeply nested dictionaries - should I look into a database or   am I just doing it wrong? Thorsten Kampe <thorsten@thorstenkampe.de> - 2011-07-31 22:04 +0200
          Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong? Philip Semanchuk <philip@semanchuk.com> - 2011-07-31 16:53 -0400
          Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong? Andrew Berg <bahamutzero8825@gmail.com> - 2011-07-31 17:26 -0500
            Re: Deeply nested dictionaries - should I look into a database or   am I just doing it wrong? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-08-01 18:29 +1200
      Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong? Chris Rebert <clp2@rebertia.com> - 2011-07-31 12:51 -0700
      Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong? Andrew Berg <bahamutzero8825@gmail.com> - 2011-07-31 15:34 -0500
        Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong? BlueBird <phil@freehackers.org> - 2011-08-02 02:17 -0700

#10611 — Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong?

On 2011.07.30 09:43 PM, Rustom Mody wrote:
> I use pyyaml for such. http://yaml.org/
> The builtin json support http://docs.python.org/library/json.html is a bit
> weaker but has the advantage of no extra install
I don't need a format to share or store the data. I'm perfectly content
to pickle the class instance for storage purposes since I don't need to
share with anything that wouldn't understand the pickled class instance.

I'm looking for pointers on design. I'm inexperienced but cautious and
am mostly wondering if there's an easier way to "format" this data or if
this approach will lead to problems.

-- 
CPython 3.2.1 | Windows NT 6.1.7601.17592 | Thunderbird 5.0
PGP/GPG Public Key ID: 0xF88E034060A78FCB

[toc] | [next] | [standalone]

#10619 — Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong?

* Andrew Berg (Sat, 30 Jul 2011 22:10:43 -0500)
> I'm looking for pointers on design. I'm inexperienced but cautious and
> am mostly wondering if there's an easier way to "format" this data or
> if this approach will lead to problems.

The "QueueItem.x264['avs']['filter']['fft3d']['ffte'])" example does not 
look right. Especially the mix of "." and "[]" references. Actually, 
dictionaries in a dictionary don't look right to me either.

The design of your data structure won't change. I would think that your 
program already mimicks it by nested class statements so you're 
duplicating the already existing class structure via dictionaries in 
order to have it available in the main class. You say that is necessary 
but I'm not convinced it is.

Another approach would be named tuples instead of dictionaries or flat 
SQL tables.

Thorsten

[toc] | [prev] | [next] | [standalone]

#10628

On 2011.07.31 02:41 AM, Thorsten Kampe wrote:
> The "QueueItem.x264['avs']['filter']['fft3d']['ffte'])" example does not 
> look right. Especially the mix of "." and "[]" references. Actually, 
> dictionaries in a dictionary don't look right to me either.
QueueItem is the class; x264 is a dictionary.

> The design of your data structure won't change. I would think that your 
> program already mimicks it by nested class statements so you're 
> duplicating the already existing class structure via dictionaries in 
> order to have it available in the main class.
Huh? I only have no nested class statements.

> Another approach would be named tuples instead of dictionaries or flat 
> SQL tables.
What would the advantage of that be?

-- 
CPython 3.2.1 | Windows NT 6.1.7601.17592 | Thunderbird 5.0
PGP/GPG Public Key ID: 0xF88E034060A78FCB

[toc] | [prev] | [next] | [standalone]

#10634 — Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong?

* Andrew Berg (Sun, 31 Jul 2011 13:36:43 -0500)
> On 2011.07.31 02:41 AM, Thorsten Kampe wrote:
> > Another approach would be named tuples instead of dictionaries or
> > flat SQL tables.
> What would the advantage of that be?

QueueItem.x264['avs']['filter']['fft3d']['ffte'] would be 
QueueItem.x264.avs.filter.fft3d.ffte. I recently "migrated" from a 
syntax of - example - datetuple[fieldpositions['tm_year'][0]] (where 
fieldpositions was a dictionary containing a list) to 
datetuple.tm_year_start which is much more readable.

The advantage of a SQL(ite) database would be simple flat tables but 
accessing them would be more difficult.

Even a INI config file structure could match your problem.

Thorsten

[toc] | [prev] | [next] | [standalone]

#10636

On Jul 31, 2011, at 4:04 PM, Thorsten Kampe wrote:

> * Andrew Berg (Sun, 31 Jul 2011 13:36:43 -0500)
>> On 2011.07.31 02:41 AM, Thorsten Kampe wrote:
>>> Another approach would be named tuples instead of dictionaries or
>>> flat SQL tables.
>> What would the advantage of that be?
> 
> QueueItem.x264['avs']['filter']['fft3d']['ffte'] would be 
> QueueItem.x264.avs.filter.fft3d.ffte. I recently "migrated" from a 
> syntax of - example - datetuple[fieldpositions['tm_year'][0]] (where 
> fieldpositions was a dictionary containing a list) to 
> datetuple.tm_year_start which is much more readable.
> 
> The advantage of a SQL(ite) database would be simple flat tables but 
> accessing them would be more difficult.
> 
> Even a INI config file structure could match your problem.

INI files are OK for lightweight use, but I find them very fragile. Since there's no specification for them, libraries don't always agree on how to read them. For instance, some libraries treat # as the comment character, and others think it is ; and others accept both. There's no standard way to specify the encoding, and, as would be critical to the OP who is nesting dicts inside of dicts, not all INI file libraries accept nested sections.

To the OP -- if you're looking to write this to disk, I recommend XML or SQLite. 

JMHO,
Philip

[toc] | [prev] | [next] | [standalone]

#10637

On 2011.07.31 03:53 PM, Philip Semanchuk wrote:
> To the OP -- if you're looking to write this to disk, I recommend XML or SQLite. 
I have a method that writes the data to disk, but at this point, I don't
see any problems with just pickling the class instance. XML might be a
good way to provide something easily read and edited by humans, though
(one major goal of the class design is to let the user provide as much
or as little info as he/she wants with a pickle or config/XML file and
provide the rest with the program's interface).  I doubt I'll use a
database for storage; it's quite practical to keep everything in memory.

-- 
CPython 3.2.1 | Windows NT 6.1.7601.17592 | Thunderbird 5.0
PGP/GPG Public Key ID: 0xF88E034060A78FCB

[toc] | [prev] | [next] | [standalone]

#10649 — Re: Deeply nested dictionaries - should I look into a database or am I just doing it wrong?

Andrew Berg wrote:

> I have a method that writes the data to disk, but at this point, I don't
> see any problems with just pickling the class instance.

Just keep in mind that if you're not careful, pickles
can end up being tied more closely that you would like
to various internal details of your program, such as
the precise names of the classes involved and which
modules they live in.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#10633

On Sun, Jul 31, 2011 at 11:36 AM, Andrew Berg <bahamutzero8825@gmail.com> wrote:
> On 2011.07.31 02:41 AM, Thorsten Kampe wrote:
<snip>
>> Another approach would be named tuples instead of dictionaries or flat
>> SQL tables.
> What would the advantage of that be?

Less punctuation noise:
QueueItem.x264.avs.filter.fft3d.ffte
vs.
QueueItem.x264['avs']['filter']['fft3d']['ffte']

It would also make clear that your sets of "keys" are static (unlike
typical dictionary usage).

Cheers,
Chris
--
http://rebertia.com

[toc] | [prev] | [next] | [standalone]

#10635

On 2011.07.31 02:51 PM, Chris Rebert wrote:
> Less punctuation noise:
> QueueItem.x264.avs.filter.fft3d.ffte
> vs.
> QueueItem.x264['avs']['filter']['fft3d']['ffte']
> 
> It would also make clear that your sets of "keys" are static (unlike
> typical dictionary usage).
I see what Thorsten meant by "named tuple" now. I was thinking of a
tuple with a name:
x = (1,2)
Your usage made me look it up and I see that it means the factory
function in the collections module. I wanted to make things look like
that, but the only way I knew how to do that would've been to a new
create blank class and assign values to its __dict__.
Named tuples look useful; I'll read the docs and see if that will work.

-- 
CPython 3.2.1 | Windows NT 6.1.7601.17592 | Thunderbird 5.0
PGP/GPG Public Key ID: 0xF88E034060A78FCB

[toc] | [prev] | [next] | [standalone]

#10710

I love named tuples, they rock for this kind of task: storing
complicated structure in a python compatible way, without too much
hassle.

And as far as load/save on disk is concerned, I simply use regular
python structure with safe eval [1]. I get all the flexibility that I
need for the file format, without the annoyance of writing a
conversion layer.

[1]: http://code.activestate.com/recipes/364469-safe-eval/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web