Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <d@davea.name>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.013
X-Spam-Evidence: '*H*': 0.97; '*S*': 0.00; 'linux,': 0.05; 'file)': 0.07; 'buffer,': 0.09; 'caching,': 0.09; 'subject:files': 0.09; 'whatever.': 0.09; 'cc:addr:python-list': 0.10; 'anyway': 0.11; 'assume': 0.11; 'cases': 0.15; 'file,': 0.15; 'mapping,': 0.16; 'measurement': 0.16; 'pipe,': 0.16; 'pipes': 0.16; 'subprocess': 0.16; 'think?': 0.16; 'wrote:': 0.17; 'pieces': 0.17; 'memory': 0.18; 'os,': 0.22; 'pipe': 0.22; 'work,': 0.22; 'nearly': 0.23; 'idea': 0.24; 'cc:2**1': 0.24; 'linux': 0.24; 'cc:addr:python.org': 0.25; 'header:In-Reply-To:1': 0.25; 'header :User-Agent:1': 0.26; '(which': 0.26; 'cc:addr:gmail.com': 0.27; 'run': 0.28; 'piece': 0.29; 'subject:size': 0.29; 'probably': 0.29; 'maybe': 0.29; 'usually': 0.30; 'file': 0.32; 'not.': 0.32; 'running': 0.32; 'allocated': 0.33; 'ram': 0.33; 'doing': 0.35; 'pm,': 0.35; 'but': 0.36; 'wanted': 0.36; 'virtual': 0.37; 'subject:: ': 0.38; 'things': 0.38; 'received:192': 0.39; 'little': 0.39; 'received:192.168': 0.40; 'subject:-': 0.40; 'your': 0.60; 'map': 0.61; 'kind': 0.61; 'between': 0.63; 'more': 0.63; 'choose': 0.65; 'frequently': 0.65; 'header:Reply-To:1': 0.68; 'fact,': 0.69; 'physical': 0.69; 'received:74.208': 0.71; 'reply-to:no real name:2**0': 0.72; 'special': 0.73; 'andrea': 0.84; 'measure.': 0.84; 'thing,': 0.84; 'why?': 0.84; 'subject:limited': 0.93
Date: Wed, 14 Nov 2012 15:57:20 -0500
From: Dave Angel <d@davea.name>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:16.0) Gecko/20121011 Thunderbird/16.0.1
MIME-Version: 1.0
To: Andrea Crotti <andrea.crotti.0@gmail.com>
Subject: Re: creating size-limited tar files
References: <mailman.3381.1352308430.27098.python-list@python.org> <CAHVvXxTH7WrfT41GDj15f9FS4eguBF2dj4Khpu_vw2TWMkvPYg@mail.gmail.com> <CAF_E5JbY71J3ix9PiXu7wvvkzc_aG9ZtdwKkuxXXAU-SFeRDsw@mail.gmail.com> <CAF_E5JYJ7Fw4b4z81454k9HwmjZ49qjx48fC_f+skcphCDNUeA@mail.gmail.com> <CAF_E5Jas3ObdX=Mhusj63R_Xcm0e1JwjPEuOgZoggON6-cttjA@mail.gmail.com> <CAF_E5JZFJ+TPmF0LUEFU8AVqbOWfJbH9Yy60CKkcrDNnWATiVg@mail.gmail.com> <CALwzidnrTcSK+Z0cGf5WpTN2sLqVgbpbt2i1DcdUFnGQYEvKQg@mail.gmail.com> <87lie4lnxr.fsf@nitrogen.i-did-not-set--mail-host-address--so-tickle-me> <CALwzidkfARHC5Twpw01xQrOWe4msWdLSH2B=5i4oMwadPyQniA@mail.gmail.com> <87fw4clg8b.fsf@nitrogen.i-did-not-set--mail-host-address--so-tickle-me> <CAF_E5JYYEQ77rEQb1tM5tDKbTTSwYNVv1An57bUKn8Y_d-VJUQ@mail.gmail.com> <CAF_E5Jbk2inQbTrurgBKP2DLjzR9=t4XXSGbafuq-E3UvJjaiw@mail.gmail.com> <50A3C287.80707@davea.name> <CAF_E5JaH332cJXko7iLuhNfxyrhE9oeUz3wMZ51XVx5CFE29kw@mail.gmail.com> <50A3C7D7.7050103@davea.name> <50A4028F.2020805@gmail.com>
In-Reply-To: <50A4028F.2020805@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:f9ASFc0mZ0Ir0OWFhbF52psKG4YCc0lLJFRfLJcvLD1 jaRd8C4/FwqwKlo1k+zM1Gyi8Kq7+khmOyxqrB7zT7NVbPmKC4 qeMSGqt7sWRYwQuNMiBjN1UAzt0AfyRiS5Fvo7sY/jVxtINyTP rCsLMF5wPqKHey7jcwWy5rVYEEv/2pWC+He+/MtCt6UpNafpl1 QVmIRVrJfgjgDPCsgcevlrfJflmV0nT4Mtl5PdmpTqkvHDbwLu Rcv5YD1uxQm1oICOCyzxaAzqnTmN6H8D0TEO6XZ/4tebr2vadP svGCf8q9rSk//CrZCYbLCv/AzOQxmtMguyXiaEZgnZX0kJFkA= =
Cc: Python <python-list@python.org>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: d@davea.name
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.3699.1352926665.27098.python-list@python.org>
Lines: 35
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1352926665 news.xs4all.nl 6853 [2001:888:2000:d::a6]:54791
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:33355

On 11/14/2012 03:43 PM, Andrea Crotti wrote:
> <SNIP>
> Anyway the only thing I wanted to understand is if using the pipes in
> subprocess is exactly the same as doing
> the Linux pipe, or not.

It's not the same thing, but you can usually assume it's close.  Other
effects will probably dominate any differences.
> 
> And any idea on how to run it in ram?
> Maybe if I create a pipe in tmpfs it might already work, what do you think?
> 
> 

In a good virtual OS, such as Linux, there's very little predictable
difference between running in RAM (which is to say reading and writing
to the swap file) or reading and writing to a file you specify.  In
fact, writing to a file can frequently be quicker, if it's sequential.

Why?  Linux is using any given piece of physical RAM to map a file, or
an allocated buffer, or shared memory, or nearly anything.  About the
only special cases are the kind of RAM that has to be locked into RAM
for hardware reasons.

Linux decides which pieces to keep in memory, whether it calls it
caching, swapping, memory mapping, or whatever.  And frequently,
attempts to "beat the system"  result in counterintuitive results.

If in doubt, measure.  But choose your measures carefully, because lots
more things will change the measurement than you might expect.


-- 

DaveA