Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #61900 > unrolled thread

Is it more CPU-efficient to read/write config file or read/write sqlite database?

Back to article view | Back to comp.lang.python

Contents

  Is it more CPU-efficient to read/write config file or read/write sqlite database? JL <lightaiyee@gmail.com> - 2013-12-14 07:29 -0800
    Re: Is it more CPU-efficient to read/write config file or read/write sqlite database? Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2013-12-14 16:40 +0100
    Re: Is it more CPU-efficient to read/write config file or read/write sqlite database? Tim Chase <python.list@tim.thechases.com> - 2013-12-14 10:15 -0600

#61900 — Is it more CPU-efficient to read/write config file or read/write sqlite database?

I have a number of python processes which communicate with each other through writing/reading config text files. The python ConfigParser is used. I am wondering if it is more CPU-efficient to switch to using sqlite database instead of using configuration files. If the software does plenty of reading/writing, is it more efficient to use config text files or sqlite database?

Thank you.

[toc] | [next] | [standalone]

#61901

On 14-12-2013 16:29, JL wrote:
> I have a number of python processes which communicate with each other through
> writing/reading config text files. The python ConfigParser is used. I am wondering if
> it is more CPU-efficient to switch to using sqlite database instead of using
> configuration files. If the software does plenty of reading/writing, is it more
> efficient to use config text files or sqlite database?
> 
> Thank you.
> 

I think you're asking the wrong question...
Both options will very likely be constrained by I/O instead of CPU (in other words: do
you know for certain that you have a CPU-bottleneck right now?)
But both of them aren't well suited for inter process communication. Especially the
"reading/writing config files" sounds particularly sketchy.

Take a look at the myriad of options for *proper* inter-process communication (if that
is what you're after). It also helps to describe your situation in more detail so we can
give better answers.

Cheers,
Irmen

[toc] | [prev] | [next] | [standalone]

#61903

On 2013-12-14 07:29, JL wrote:
> I have a number of python processes which communicate with each
> other through writing/reading config text files. The python
> ConfigParser is used. I am wondering if it is more CPU-efficient to
> switch to using sqlite database instead of using configuration
> files. If the software does plenty of reading/writing, is it more
> efficient to use config text files or sqlite database?

I'm pretty sure that the CPU aspect doesn't really play into things.
A few thoughts:

+ You'll be I/O bound most of the time.  Even if you used a ramdisk
  to reduce disk access delays, accessing multiple .txt files requires
  the OS to do permission-checking each time, while a single sqlite
  file gets checked once upon opening the DB initially.
+ text-files are fragile unless you take extra pains to keep things
  atomic
+ sqlite guarantee* atomicity, so you either see all-or-nothing
+ sqlite is also very efficient for querying
+ sticking with plain-text config files is just asking for some sort
  of race-condition or partial-file issue to come up
+ sqlite may give you less CPU load is just an added benefit


-tkc


* well, except on NFS shares and other places where file-locking is
  unreliable

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web