Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Tim Chase <python.list@tim.thechases.com>
Newsgroups: comp.lang.python
Subject: Re: Storing a big amount of path names
Date: Thu, 11 Feb 2016 19:13:32 -0600
Lines: 27
Message-ID: <mailman.63.1455239783.22075.python-list@python.org>
References: <n9j94f$712$1@gioia.aioe.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
In-Reply-To: <n9j94f$712$1@gioia.aioe.org>
Precedence: list
Xref: csiph.com comp.lang.python:102837

On 2016-02-12 00:31, Paulo da Silva wrote:
> What is the best (shortest memory usage) way to store lots of
> pathnames in memory where:
> 
> 1. Path names are pathname=(dirname,filename)
> 2. There many different dirnames but much less than pathnames
> 3. dirnames have in general many chars
> 
> The idea is to share the common dirnames.

Well, you can create a dict that has dirname->list(filenames) which
will reduce the dirname to a single instance.  You could store that
dict in the class, shared by all of the instances, though that starts
to pick up a code-smell.

But unless you're talking about an obscenely large number of
dirnames & filenames, or a severely resource-limited machine, just
use the default built-ins.  If you start to push the boundaries of
system resources, then I'd try the "anydbm" module or use the
"shelve" module to marshal them out to disk.  Finally, you *could*
create an actual sqlite database on disk if size really does exceed
reasonable system specs.

-tkc