Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #15115

save tuple of simple data types to disk (low memory foot print)

Show all headers | View raw

Hi,

I would like to save many dicts with a fixed amount of keys
tuples to a file  in a memory efficient manner (no random, but only
sequential access is required)

As the keys are the same for each entry  I considered converting them to
tuples.

The tuples contain only strings, ints (long ints) and floats (double)
and the data types for each position within the tuple are fixed.

The fastest and simplest way is to pickle the data or to use json.
Both formats however are not that optimal.


I could store ints and floats with pack. As strings have variable length
I'm not sure how to save them efficiently
(except adding a length first and then the string.

Is there already some 'standard' way or standard library to store
such data efficiently?

Thanks in advance for any suggestion.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

save tuple of simple data types to disk (low memory foot print) Gelonida N <gelonida@gmail.com> - 2011-10-28 22:47 +0200
  Re: save tuple of simple data types to disk (low memory foot print) Roy Smith <roy@panix.com> - 2011-10-28 19:08 -0400
    Re: save tuple of simple data types to disk (low memory foot print) Gelonida N <gelonida@gmail.com> - 2011-10-29 18:47 +0200
  Re: save tuple of simple data types to disk (low memory foot print) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-10-29 01:00 +0000
    Re: save tuple of simple data types to disk (low memory foot print) Gelonida N <gelonida@gmail.com> - 2011-10-29 18:44 +0200
    Re: save tuple of simple data types to disk (low memory foot print) Tim Chase <python.list@tim.thechases.com> - 2011-10-29 12:47 -0500

csiph-web