Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Peter Otten <__peter__@web.de>
Newsgroups: comp.lang.python
Subject: Re: In Python 3, how to append a nested dictionary to a shelve file with the added difficulty of using a for loop?
Date: Tue, 22 Dec 2015 01:23:51 +0100
Organization: None
Lines: 148
Message-ID: <mailman.50.1450743843.2237.python-list@python.org>
References: <CAOA=+NtE6p4XXnWr6TsUgc-8ewtDu5Xpy5ZcnZX=p5G+3f44DA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7Bit
User-Agent: KNode/4.13.3
Precedence: list
Xref: csiph.com comp.lang.python:100712

Aaron Christensen wrote:

> Hello,
> 
> I am trying to figure out how to populate a shelve file with a nested
> dictionary.
> 
> These are my requirements:
> 
> -Create shelve file called people.db
> -Append the shelve file with new people (person_1, person_2, etc.).
> -Use a for loop to iterate through 'attributes' so that I do not need to
> write out the lengthy code line by line to populate to the shelve file.
> -Need to reference shelve file data for future use
> 
> Here is the key/value format that I would like to append to the shelve
> file.
> 
> person_1 = { 'name': 'Bob', 'type': 'employee', 'attributes':
>         [{'game': 'basketball', 'high score': '100', 'time': '3.34'},
>         {'game': 'bridge', 'high score': '10', 'time': '30.34'},
>         {'game': 'foosball', 'high score': '2', 'time': '24'}]
>         '''
>         50+ other attributes
>         '''
>         }
> 
> # Example: s['person_1]['attributes'][2]['time'] would call out '24'.
> # 's' is from 's = shelve.open('people')'
> I have a dictionary dictPeople.py file (created using pprint() that
> contains the information of person_1, etc. And I am extracting only a
> small percentage of the data that is needed.
> 
> I have tried the following, but I get an invalid key error.
> 
> import shelve, dictPeople
> s = shelve.open('people')
> person = 'person_1'
> s[person]['name'] = dictPeople.person_1['name']
> s[person]['type'] = dictPeople.person_1['type']
> # I need to use this for loop because there are 50+ attributes.
> for attribute in range(0, len(dictPeople['attributes'][attribute])):
> s[person]['attributes'][attribute]['game'] = \
>     dictPeople['attributes'][attribute]['game']
> s[person]['attributes'][attribute]['high score'] = \
>     dictPeople['attributes'][attribute]['high score']
> s[person]['attributes'][attribute]['time'] = \
>     dictPeople['attributes'][attribute]['time']
> It turns out, I get the key error because I am not allowed to reference a
> key/value pair the same way that I can with a dictionary. However, the
> crazy thing is that I can call values in the db file using the same exact
> format when trying to write.
> 
> For example: I can read data from the .db file using:
> 
> x = s['person_1']['name']
> print(x)
> BUT! I cannot write to that .db file using that exact format or I get an
> invalid key error. Makes no sense!
> 
> s['person_1']['name'] = 'Bob'
> # Returns invalid key entry.  Makes no sense.
> Therefore, I tried to extract data and populate the db file using the
> following:
> 
> s[person] = {   'name': dictPeople.person_1['name'],
>             'type': dictPeople.person_1['type'],
> for attribute in range(0, len(dictPeople['attributes'][attribute])):
>     ['game':  dictPeople['attributes'][attribute]['game'],
>     'high score': dictPeople['attributes'][attribute]['high score'],
>     'time': dictPeople['attributes'][attribute]['time']]
> 
> }
> But, this obvously doesn't work because of the for loop. How can I do
> this?
> 
> -I am trying to figure out how to extract data from the dictionary
> dictPeople.py file and store it in the people.db file.
> -I am trying to add new people to the people.db file as more people become
> available.
> -I need to use the for loop because of the 50+ attributes I need to add.
> -My future steps would be to modify some of the data values that I extract
> and used to populate the people.db file, but my first step is to, at
> least, extract, and then populate the people.db file.
> 
> Any help or guidance is greatly appreciated.  Thank you for your time and
> reading my question.

You don't need these loops. The problem is that when you read the dict from 
the db

db = shelve.open(...)
person = db["person_1"]

person is a Python object completely independent of the shelve and the 
shelve has no way to trace changes to that object:

$ python3
Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> db = shelve.open("tmp.shelve")
>>> person_1 = { 'name': 'Bob', 'type': 'employee', 'attributes':
...         [{'game': 'basketball', 'high score': '100', 'time': '3.34'},
...         {'game': 'bridge', 'high score': '10', 'time': '30.34'},
...         {'game': 'foosball', 'high score': '2', 'time': '24'}]
... }
>>> db["person_1"] = person_1
>>> db["person_1"]["age"] = 42
>>> db.close()
>>> db = shelve.open("tmp.shelve")
>>> db["person_1"]
{'name': 'Bob', 'type': 'employee', 'attributes': [{'time': '3.34', 'high 
score': '100', 'game': 'basketball'}, {'time': '30.34', 'high score': '10', 
'game': 'bridge'}, {'time': '24', 'high score': '2', 'game': 'foosball'}]}

When you look at the data you see that there is no "age" key. The straight-
forward fix is to always use an assignment 

db[key] = new_value

when you want a change:

>>> person = db["person_1"]
>>> person["age"] = 42
>>> db["person_1"] = person
>>> db.close()

That way the change will survive:

>>> db = shelve.open("tmp.shelve")
>>> db["person_1"]["age"]
42

Alternatively you can open the shelve with writeback=True:

>>> db = shelve.open("tmp.shelve", writeback=True)
>>> db["person_1"]["height"] = 2.0
>>> db.close()
>>> db = shelve.open("tmp.shelve", writeback=True)
>>> db["person_1"]["height"]
2.0

That way every value will be written back (even when you don't modify it) 
and therefore all mutations persist, but you pay for the convenience with 
superfluous caching and writing.