stable object serialization to text file

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<koch.mate@me.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.081
X-Spam-Evidence	'H': 0.84; 'S': 0.00; 'subject:object': 0.07; 'python': 0.08; "hasn't": 0.13; 'subject:file': 0.13; 'received:172.18.0': 0.16; 'unordered': 0.16; 'thanks,': 0.18; 'subject:skip:s 10': 0.18; 'suggest': 0.20; 'objects,': 0.23; 'received:213.165': 0.24; 'developing': 0.25; 'module': 0.26; 'load': 0.26; 'all,': 0.27; "i'm": 0.27; 'problem': 0.29; 'efficiently': 0.30; 'far,': 0.30; '(e.g.': 0.30; 'app': 0.31; 'version': 0.31; "isn't": 0.32; 'to:addr:python-list': 0.33; 'there': 0.33; 'force': 0.34; 'mostly': 0.34; 'anything': 0.34; 'numbers.': 0.34; 'stores': 0.34; 'from:charset:iso-8859-1': 0.34; 'subject:text': 0.35; 'file': 0.35; 'header:Received:8': 0.36; 'charset:us-ascii': 0.36; 'but': 0.37; 'easiest': 0.38; 'created': 0.38; 'possible.': 0.39; "i'd": 0.39; 'change': 0.40; 'to:addr:python.org': 0.40; 'data': 0.40; 'course,': 0.61; 'kind': 0.62; 'header:Message-Id:1': 0.62; 'texts': 0.73; 'database.': 0.74; 'received:(helo localhost)': 0.77; 'dict,': 0.84; 'mate': 0.84; 'received:hu': 0.93
X-Virus-Scanned	by amavisd-new-2.6.4 (20090625) at allied-visions.de
From	Máté Koch <koch.mate@me.com>
Content-Type	text/plain; charset=us-ascii
Content-Transfer-Encoding	quoted-printable
Subject	stable object serialization to text file
Date	Wed, 11 Jan 2012 21:16:45 +0100
To	python-list@python.org
Mime-Version	1.0 (Apple Message framework v1251.1)
X-Mailer	Apple Mail (2.1251.1)
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.12
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.4680.1326369724.27778.python-list@python.org> (permalink)
Lines	20
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1326369724 news.xs4all.nl 6855 [2001:888:2000:d::a6]:52483
X-Complaints-To	abuse@xs4all.nl
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:18873

Show key headers only | View raw

Hello All,

I'm developing an app which stores the data in file system database. The data in my case consists of large python objects, mostly dicts, containing texts and numbers. The easiest way to dump and load them would be pickle, but I have a problem with it: I want to keep the data in version control, and I would like to use it as efficiently as possible. Is it possible to force pickle to store the otherwise unordered (e.g. dictionary) data in a kind of ordered way, so that if I dump a large dict, then change 1 tiny thing in it and dump again, the diff of the former and the new file will be minimal?

If pickle is not the best choice for me, can you suggest anything else? (If there isn't any solution for it so far, I will write the module of course, but first I'd like to look around and make sure it hasn't been created yet.)

Thanks,

Mate

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

stable object serialization to text file Máté Koch <koch.mate@me.com> - 2012-01-11 21:16 +0100
  Re: stable object serialization to text file K Richard Pixley <rich@noir.com> - 2012-01-12 17:08 -0800

csiph-web