Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #100712

Re: In Python 3, how to append a nested dictionary to a shelve file with the added difficulty of using a for loop?

Show all headers | View raw

Aaron Christensen wrote:

> Hello,
> 
> I am trying to figure out how to populate a shelve file with a nested
> dictionary.
> 
> These are my requirements:
> 
> -Create shelve file called people.db
> -Append the shelve file with new people (person_1, person_2, etc.).
> -Use a for loop to iterate through 'attributes' so that I do not need to
> write out the lengthy code line by line to populate to the shelve file.
> -Need to reference shelve file data for future use
> 
> Here is the key/value format that I would like to append to the shelve
> file.
> 
> person_1 = { 'name': 'Bob', 'type': 'employee', 'attributes':
>         [{'game': 'basketball', 'high score': '100', 'time': '3.34'},
>         {'game': 'bridge', 'high score': '10', 'time': '30.34'},
>         {'game': 'foosball', 'high score': '2', 'time': '24'}]
>         '''
>         50+ other attributes
>         '''
>         }
> 
> # Example: s['person_1]['attributes'][2]['time'] would call out '24'.
> # 's' is from 's = shelve.open('people')'
> I have a dictionary dictPeople.py file (created using pprint() that
> contains the information of person_1, etc. And I am extracting only a
> small percentage of the data that is needed.
> 
> I have tried the following, but I get an invalid key error.
> 
> import shelve, dictPeople
> s = shelve.open('people')
> person = 'person_1'
> s[person]['name'] = dictPeople.person_1['name']
> s[person]['type'] = dictPeople.person_1['type']
> # I need to use this for loop because there are 50+ attributes.
> for attribute in range(0, len(dictPeople['attributes'][attribute])):
> s[person]['attributes'][attribute]['game'] = \
>     dictPeople['attributes'][attribute]['game']
> s[person]['attributes'][attribute]['high score'] = \
>     dictPeople['attributes'][attribute]['high score']
> s[person]['attributes'][attribute]['time'] = \
>     dictPeople['attributes'][attribute]['time']
> It turns out, I get the key error because I am not allowed to reference a
> key/value pair the same way that I can with a dictionary. However, the
> crazy thing is that I can call values in the db file using the same exact
> format when trying to write.
> 
> For example: I can read data from the .db file using:
> 
> x = s['person_1']['name']
> print(x)
> BUT! I cannot write to that .db file using that exact format or I get an
> invalid key error. Makes no sense!
> 
> s['person_1']['name'] = 'Bob'
> # Returns invalid key entry.  Makes no sense.
> Therefore, I tried to extract data and populate the db file using the
> following:
> 
> s[person] = {   'name': dictPeople.person_1['name'],
>             'type': dictPeople.person_1['type'],
> for attribute in range(0, len(dictPeople['attributes'][attribute])):
>     ['game':  dictPeople['attributes'][attribute]['game'],
>     'high score': dictPeople['attributes'][attribute]['high score'],
>     'time': dictPeople['attributes'][attribute]['time']]
> 
> }
> But, this obvously doesn't work because of the for loop. How can I do
> this?
> 
> -I am trying to figure out how to extract data from the dictionary
> dictPeople.py file and store it in the people.db file.
> -I am trying to add new people to the people.db file as more people become
> available.
> -I need to use the for loop because of the 50+ attributes I need to add.
> -My future steps would be to modify some of the data values that I extract
> and used to populate the people.db file, but my first step is to, at
> least, extract, and then populate the people.db file.
> 
> Any help or guidance is greatly appreciated.  Thank you for your time and
> reading my question.

You don't need these loops. The problem is that when you read the dict from 
the db

db = shelve.open(...)
person = db["person_1"]

person is a Python object completely independent of the shelve and the 
shelve has no way to trace changes to that object:

$ python3
Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> db = shelve.open("tmp.shelve")
>>> person_1 = { 'name': 'Bob', 'type': 'employee', 'attributes':
...         [{'game': 'basketball', 'high score': '100', 'time': '3.34'},
...         {'game': 'bridge', 'high score': '10', 'time': '30.34'},
...         {'game': 'foosball', 'high score': '2', 'time': '24'}]
... }
>>> db["person_1"] = person_1
>>> db["person_1"]["age"] = 42
>>> db.close()
>>> db = shelve.open("tmp.shelve")
>>> db["person_1"]
{'name': 'Bob', 'type': 'employee', 'attributes': [{'time': '3.34', 'high 
score': '100', 'game': 'basketball'}, {'time': '30.34', 'high score': '10', 
'game': 'bridge'}, {'time': '24', 'high score': '2', 'game': 'foosball'}]}

When you look at the data you see that there is no "age" key. The straight-
forward fix is to always use an assignment 

db[key] = new_value

when you want a change:

>>> person = db["person_1"]
>>> person["age"] = 42
>>> db["person_1"] = person
>>> db.close()

That way the change will survive:

>>> db = shelve.open("tmp.shelve")
>>> db["person_1"]["age"]
42

Alternatively you can open the shelve with writeback=True:

>>> db = shelve.open("tmp.shelve", writeback=True)
>>> db["person_1"]["height"] = 2.0
>>> db.close()
>>> db = shelve.open("tmp.shelve", writeback=True)
>>> db["person_1"]["height"]
2.0

That way every value will be written back (even when you don't modify it) 
and therefore all mutations persist, but you pay for the convenience with 
superfluous caching and writing.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: In Python 3, how to append a nested dictionary to a shelve file with the added difficulty of using a for loop? Peter Otten <__peter__@web.de> - 2015-12-22 01:23 +0100

csiph-web