Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #30427 > unrolled thread

RE: [python-list] python application file format

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  RE: [python-list] python application file format "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-09-28 20:34 +0000
    Re: [python-list] python application file format Ramchandra Apte <maniandram01@gmail.com> - 2012-09-29 06:35 -0700
    Re: [python-list] python application file format Ramchandra Apte <maniandram01@gmail.com> - 2012-09-29 06:35 -0700

#30427 — RE: [python-list] python application file format

Benjamin Jessup wrote:
> Hello all,
> 
> What do people recommend for a file format for a python desktop
> application? Data is complex with 100s/1000s of class instances, which
> reference each other.
> 
> Write the file with struct module? (Rebuild object pointers, safe,
> compact, portable, not expandable without reserved space)
> 
> Use cPickle with a module/class whitelist? (Can't easily port, not
> entirely safe, compact enough, expandable)
> 
> Use JSON or similar? (Rebuild object pointers, portable, expandable, size?)
> 
> Any advice is greatly appreciated!

I would think your options are pickle, json or database (either sqlite or 
something like Postgres). I am unfamiliar with the struct module so I 
cannot comment on its applicability.

I would guess that your data would be best saved by using a sqlite 
database. Your biggest problem might be how the different classes are 
referencing each other. If you are using identifiers then any of these 
options will probably work. If you are using aggregation then I know
that pickle will work (at least for built-in types). JSON will
keep the structure but duplicate elements.


>>> a = [ 1,2,3 ]
>>> b = [ 'a', 'b', 'c' ]
>>> a.append( b )
>>> e = [ a,b ]
>>> s = json.dumps( e ) 
>>> eret = json.loads( s )
>>> id(eret[0][3]), id(eret[1]) # Same result for json and simplejson
(329443808, 327677272) 
>>> eret[0][3].append( 'o')
>>> eret[0][3], eret[1]
([u'a', u'b', u'c', 'o'], [u'a', u'b', u'c'])

So pickle will be your easiest option, but I am not sure how well it
will scale with a large number items. Using sqlite/db should scale well
but it will take you longer/more effort to create a system for converting 
your objects to and from the DB.



This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

[toc] | [next] | [standalone]

#30488

On Saturday, 29 September 2012 02:05:07 UTC+5:30, Prasad, Ramit  wrote:
> Benjamin Jessup wrote:
> 
> > Hello all,
> 
> > 
> 
> > What do people recommend for a file format for a python desktop
> 
> > application? Data is complex with 100s/1000s of class instances, which
> 
> > reference each other.
> 
> > 
> 
> > Write the file with struct module? (Rebuild object pointers, safe,
> 
> > compact, portable, not expandable without reserved space)
> 
> > 
> 
> > Use cPickle with a module/class whitelist? (Can't easily port, not
> 
> > entirely safe, compact enough, expandable)
> 
> > 
> 
> > Use JSON or similar? (Rebuild object pointers, portable, expandable, size?)
> 
> > 
> 
> > Any advice is greatly appreciated!
> 
> 
> 
> I would think your options are pickle, json or database (either sqlite or 
> 
> something like Postgres). I am unfamiliar with the struct module so I 
> 
> cannot comment on its applicability.
> 
> 
> 
> I would guess that your data would be best saved by using a sqlite 
> 
> database. Your biggest problem might be how the different classes are 
> 
> referencing each other. If you are using identifiers then any of these 
> 
> options will probably work. If you are using aggregation then I know
> 
> that pickle will work (at least for built-in types). JSON will
> 
> keep the structure but duplicate elements.
> 
> 
> 
> 
> 
> >>> a = [ 1,2,3 ]
> 
> >>> b = [ 'a', 'b', 'c' ]
> 
> >>> a.append( b )
> 
> >>> e = [ a,b ]
> 
> >>> s = json.dumps( e ) 
> 
> >>> eret = json.loads( s )
> 
> >>> id(eret[0][3]), id(eret[1]) # Same result for json and simplejson
> 
> (329443808, 327677272) 
> 
> >>> eret[0][3].append( 'o')
> 
> >>> eret[0][3], eret[1]
> 
> ([u'a', u'b', u'c', 'o'], [u'a', u'b', u'c'])
> 
> 
> 
> So pickle will be your easiest option, but I am not sure how well it
> 
> will scale with a large number items. Using sqlite/db should scale well
> 
> but it will take you longer/more effort to create a system for converting 
> 
> your objects to and from the DB.
> 
> 
> 
> 
> 
> 
> 
> This email is confidential and subject to important disclaimers and
> 
> conditions including on offers for the purchase or sale of
> 
> securities, accuracy and completeness of information, viruses,
> 
> confidentiality, legal privilege, and legal entity disclaimers,
> 
> available at http://www.jpmorgan.com/pages/disclosures/email.

Guess I shouldn't read or rely your posts (virus'es could be in them) :-D

[toc] | [prev] | [next] | [standalone]

#30489

On Saturday, 29 September 2012 02:05:07 UTC+5:30, Prasad, Ramit  wrote:
> Benjamin Jessup wrote:
> 
> > Hello all,
> 
> > 
> 
> > What do people recommend for a file format for a python desktop
> 
> > application? Data is complex with 100s/1000s of class instances, which
> 
> > reference each other.
> 
> > 
> 
> > Write the file with struct module? (Rebuild object pointers, safe,
> 
> > compact, portable, not expandable without reserved space)
> 
> > 
> 
> > Use cPickle with a module/class whitelist? (Can't easily port, not
> 
> > entirely safe, compact enough, expandable)
> 
> > 
> 
> > Use JSON or similar? (Rebuild object pointers, portable, expandable, size?)
> 
> > 
> 
> > Any advice is greatly appreciated!
> 
> 
> 
> I would think your options are pickle, json or database (either sqlite or 
> 
> something like Postgres). I am unfamiliar with the struct module so I 
> 
> cannot comment on its applicability.
> 
> 
> 
> I would guess that your data would be best saved by using a sqlite 
> 
> database. Your biggest problem might be how the different classes are 
> 
> referencing each other. If you are using identifiers then any of these 
> 
> options will probably work. If you are using aggregation then I know
> 
> that pickle will work (at least for built-in types). JSON will
> 
> keep the structure but duplicate elements.
> 
> 
> 
> 
> 
> >>> a = [ 1,2,3 ]
> 
> >>> b = [ 'a', 'b', 'c' ]
> 
> >>> a.append( b )
> 
> >>> e = [ a,b ]
> 
> >>> s = json.dumps( e ) 
> 
> >>> eret = json.loads( s )
> 
> >>> id(eret[0][3]), id(eret[1]) # Same result for json and simplejson
> 
> (329443808, 327677272) 
> 
> >>> eret[0][3].append( 'o')
> 
> >>> eret[0][3], eret[1]
> 
> ([u'a', u'b', u'c', 'o'], [u'a', u'b', u'c'])
> 
> 
> 
> So pickle will be your easiest option, but I am not sure how well it
> 
> will scale with a large number items. Using sqlite/db should scale well
> 
> but it will take you longer/more effort to create a system for converting 
> 
> your objects to and from the DB.
> 
> 
> 
> 
> 
> 
> 
> This email is confidential and subject to important disclaimers and
> 
> conditions including on offers for the purchase or sale of
> 
> securities, accuracy and completeness of information, viruses,
> 
> confidentiality, legal privilege, and legal entity disclaimers,
> 
> available at http://www.jpmorgan.com/pages/disclosures/email.

Guess I shouldn't read or rely your posts (virus'es could be in them) :-D

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web