Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #102834

Re: Storing a big amount of path names

Path csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From Chris Angelico <rosuav@gmail.com>
Newsgroups comp.lang.python
Subject Re: Storing a big amount of path names
Date Fri, 12 Feb 2016 11:39:32 +1100
Lines 24
Message-ID <mailman.60.1455237579.22075.python-list@python.org> (permalink)
References <n9j94f$712$1@gioia.aioe.org>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
X-Trace news.uni-berlin.de BnfYKUUaRo8JkL8HazC+cQTJAga5bD+OTldwi5PnhpHA==
Return-Path <rosuav@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.022
X-Spam-Evidence '*H*': 0.96; '*S*': 0.00; 'received:209.85.223': 0.03; 'cc:addr:python-list': 0.09; 'throw': 0.09; 'stored': 0.10; '(shortest': 0.16; '2016': 0.16; 'dirnames': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'pathnames': 0.16; 'paulo': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'wrote:': 0.16; 'memory': 0.17; 'cc:2**0': 0.20; 'cc:addr:python.org': 0.20; 'am,': 0.23; 'feb': 0.23; 'header:In-Reply-To:1': 0.24; 'fri,': 0.27; 'message- id:@mail.gmail.com': 0.27; 'idea': 0.28; 'objects': 0.29; 'common': 0.33; 'ram': 0.33; 'received:google.com': 0.35; 'nothing.': 0.35; 'path': 0.35; 'but': 0.36; 'there': 0.36; 'received:209.85': 0.36; 'subject:: ': 0.37; 'being': 0.37; '12,': 0.37; 'virtual': 0.38; 'received:209': 0.38; 'names': 0.38; 'stuff': 0.38; 'skip:p 20': 0.38; 'still': 0.40; 'share': 0.61; 'more': 0.63; 'different': 0.63; 'where:': 0.66; 'computers': 0.70; 'obvious': 0.76; 'chrisa': 0.84; 'have?': 0.84; 'to:none': 0.91; 'afford': 0.91
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=2AP9x3ji4DMLz+lZl+p9Bt9uwRUWyXcNK4RcXlGDgEI=; b=BcIV8n1LbnPmxZczmDy6vvfej8txsGc5I1Gb/1w7W7v/sq5svpvODwJRXpMo1EwAUt 8/m9hhP6Bi2SrHiCrnQv/Ueh3lujIlPOISuJdizeNabkx4Mh5+rCmRfhjZL46JbqUM+b RFon0mret09JbWaC7CUtMMVFvKMkcbgIdCzqLKcIH2Y+qwj8q/yk19Dqzz8DVAd11lg5 LfYUPROuGetoG10m6pGB7v422bI/pHRhE3sA5O5+pKCYTMVqr3d6v0DUxFOPbISDF6ge qvfy5Ifh4CVaIB/qiICBpjOJD4YoItGhAt2f/ftBS10VfhfANt8tG933vzaZWMzskngy gjRA==
X-Google-DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:cc:content-type; bh=2AP9x3ji4DMLz+lZl+p9Bt9uwRUWyXcNK4RcXlGDgEI=; b=nF4/ovoh6fHrqi5hg8pDVHBcsS+P0l202n9qpZidTyEhPEYl6tUnVDUyGdpzcnquf6 TYHL1KKLXfMIkvKOotcuYkYvT5b9W6dce6bRlH0hKy1RZ/sXFj5ruWLYVqYLodyDbV5q n2Ec2AL8BIa1TqZpwVsrHg53PTWUjK9fxIy4s6V8a1bwZJcrjcJ82dSrhKVEtPFcRAfz NgDZGM3DijaXkUnhz/ob0zH9J0l4kp2rl1H+cWOmneONJrevIEvSIib/k2qGey95IKXA kvi1OF7YxuwTaApOckb6wSMnXUf2EcngLW4wqGSudAPXt+aEjm2JQQbMnl1tNggaSbms dDeA==
X-Gm-Message-State AG10YOSN+3dMyxAt+txtCHy+Dyh/nAZhBbfKOw+lRBiWsaMpIqg9tjjXGv+kCgQUUhnanWxDOe76HTfs6M3e1g==
X-Received by 10.107.47.162 with SMTP id v34mr44615288iov.19.1455237572715; Thu, 11 Feb 2016 16:39:32 -0800 (PST)
In-Reply-To <n9j94f$712$1@gioia.aioe.org>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.21rc2
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref csiph.com comp.lang.python:102834

Show key headers only | View raw

On Fri, Feb 12, 2016 at 11:31 AM, Paulo da Silva
<p_s_d_a_s_i_l_v_a_ns@netcabo.pt> wrote:
> What is the best (shortest memory usage) way to store lots of pathnames
> in memory where:
>
> 1. Path names are pathname=(dirname,filename)
> 2. There many different dirnames but much less than pathnames
> 3. dirnames have in general many chars
>
> The idea is to share the common dirnames.
>
> More realistically not only the pathnames are stored but objects each
> object being a MyFile containing
> self.name - <base name>
> getPathname(self) - <full pathname>
> other stuff

Just store them in the most obvious way, and don't worry about memory
usage. How many path names are you likely to have? A million? You can
still afford to have 1KB pathnames and it'll take up no more than a
gigabyte of RAM - and most computers throw around gigs of virtual
memory like it's nothing.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Storing a big amount of path names Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-02-12 00:31 +0000
  Re: Storing a big amount of path names Chris Angelico <rosuav@gmail.com> - 2016-02-12 11:39 +1100
  Re: Storing a big amount of path names Ben Finney <ben+python@benfinney.id.au> - 2016-02-12 11:44 +1100
  Re: Storing a big amount of path names Tim Chase <python.list@tim.thechases.com> - 2016-02-11 19:13 -0600
    Re: Storing a big amount of path names Rob Gaddi <rgaddi@highlandtechnology.invalid> - 2016-02-12 02:17 +0000
  Re: Storing a big amount of path names MRAB <python@mrabarnett.plus.com> - 2016-02-12 03:13 +0000
  Re: Storing a big amount of path names Chris Angelico <rosuav@gmail.com> - 2016-02-12 14:49 +1100
    Re: Storing a big amount of path names Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-02-12 04:15 +0000
      Re: Storing a big amount of path names Chris Angelico <rosuav@gmail.com> - 2016-02-12 15:23 +1100
        Re: Storing a big amount of path names Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-02-12 04:45 +0000
          Re: Storing a big amount of path names Chris Angelico <rosuav@gmail.com> - 2016-02-12 16:02 +1100
            Re: Storing a big amount of path names Paulo da Silva <p_s_d_a_s_i_l_v_a_ns@netcabo.pt> - 2016-02-12 05:49 +0000
            Re: Storing a big amount of path names Steven D'Aprano <steve@pearwood.info> - 2016-02-12 16:51 +1100
        Re: Storing a big amount of path names Rob Gaddi <rgaddi@highlandtechnology.invalid> - 2016-02-12 17:05 +0000
          Re: Storing a big amount of path names Chris Angelico <rosuav@gmail.com> - 2016-02-13 04:18 +1100
          Re: Storing a big amount of path names Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-02-12 21:37 +0000
          Re: Storing a big amount of path names Ben Finney <ben+python@benfinney.id.au> - 2016-02-13 08:49 +1100
          Re: Storing a big amount of path names Matt Wheeler <funkyhat@gmail.com> - 2016-02-12 23:31 +0000
            Re: Storing a big amount of path names mkondrashin@gmail.com - 2016-02-13 12:19 -0800
  Re: Storing a big amount of path names srinivas devaki <mr.eightnoteight@gmail.com> - 2016-02-12 11:46 +0530

csiph-web