Re: creating size-limited tar files

Path	csiph.com!usenet.pasdenom.info!dedibox.gegeweb.org!gegeweb.eu!nntpfeed.proxad.net!proxad.net!feeder1-2.proxad.net!usenet-fr.net!nerim.net!novso.com!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<andrea.crotti.0@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.005
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'skip:[ 20': 0.03; 'sys': 0.05; 'skip:/ 10': 0.07; "'w')": 0.09; '/dev/null': 0.09; 'subject:files': 0.09; 'cc:addr:python-list': 0.10; 'pipe,': 0.16; 'subprocess': 0.16; 'import': 0.21; 'pipe': 0.22; 'cc:2**0': 0.23; 'cc:addr:python.org': 0.25; 'header:In-Reply-To:1': 0.25; 'message-id:@mail.gmail.com': 0.27; 'cat': 0.29; 'subject:size': 0.29; 'system,': 0.32; 'received:google.com': 0.34; 'doing': 0.35; 'received:209.85': 0.35; 'skip:p 20': 0.36; 'received:209': 0.37; 'subject:: ': 0.38; 'where': 0.40; 'subject:-': 0.40; 'header:Received:5': 0.40; 'real': 0.61; 'nice,': 0.84; 'subject:limited': 0.93
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Cxaq+IlWOC3IgEs55jQyBwYuY51JrX6kcY/GQUJ+Tzc=; b=vhPDBgT/SQnqWl0LoIwc/rXKdbAsMyAyCnHLcTZELDMHOOS/NbVlWoX/XygzWMulBD c5w40pBEp5bSPc9moKerGGBIzjmDzONJsX6ktRlOhXThWokFp1glc3QiVV8DCo1YH1bc pzbENuPBswSOAFFqsPwQ1kUGjw/tiYZQGxMPXKJwK70RFDpMo/dyXpj+zf4JYTi8WJjj aGVnz+SrBk5D4TdMSkXn02+aicQhhe5Yd/zALgHwL0zS0uVrp3vfEvsDQIEN8ZGbAcPy payunxXtFVWz5ollb8KkIsDGhlEndQ9iI4UFnhW2sxlI/wGYW5LAzMiitk/8wEMyN3bg B6eA==
MIME-Version	1.0
In-Reply-To	<CAF_E5JYYEQ77rEQb1tM5tDKbTTSwYNVv1An57bUKn8Y_d-VJUQ@mail.gmail.com>
References	<mailman.3381.1352308430.27098.python-list@python.org> <509ab0fa$0$6636$9b4e6d93@newsspool2.arcor-online.net> <roy-2C9B4D.15322307112012@news.panix.com> <509AD812.2060605@gmail.com> <CAHVvXxTH7WrfT41GDj15f9FS4eguBF2dj4Khpu_vw2TWMkvPYg@mail.gmail.com> <CAF_E5JbY71J3ix9PiXu7wvvkzc_aG9ZtdwKkuxXXAU-SFeRDsw@mail.gmail.com> <CAF_E5JYJ7Fw4b4z81454k9HwmjZ49qjx48fC_f+skcphCDNUeA@mail.gmail.com> <CAF_E5Jas3ObdX=Mhusj63R_Xcm0e1JwjPEuOgZoggON6-cttjA@mail.gmail.com> <CAF_E5JZFJ+TPmF0LUEFU8AVqbOWfJbH9Yy60CKkcrDNnWATiVg@mail.gmail.com> <CALwzidnrTcSK+Z0cGf5WpTN2sLqVgbpbt2i1DcdUFnGQYEvKQg@mail.gmail.com> <87lie4lnxr.fsf@nitrogen.i-did-not-set--mail-host-address--so-tickle-me> <CALwzidkfARHC5Twpw01xQrOWe4msWdLSH2B=5i4oMwadPyQniA@mail.gmail.com> <87fw4clg8b.fsf@nitrogen.i-did-not-set--mail-host-address--so-tickle-me> <CAF_E5JYYEQ77rEQb1tM5tDKbTTSwYNVv1An57bUKn8Y_d-VJUQ@mail.gmail.com>
Date	Wed, 14 Nov 2012 15:56:28 +0000
Subject	Re: creating size-limited tar files
From	andrea crotti <andrea.crotti.0@gmail.com>
To	Kushal Kumaran <kushal.kumaran+python@gmail.com>
Content-Type	text/plain; charset=ISO-8859-1
Cc	Python <python-list@python.org>
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.3688.1352908591.27098.python-list@python.org> (permalink)
Lines	31
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1352908591 news.xs4all.nl 6920 [2001:888:2000:d::a6]:53131
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:33342

Show key headers only | View raw

Ok this is all very nice, but:

[andrea@andreacrotti tar_baller]$ time python2 test_pipe.py > /dev/null

real	0m21.215s
user	0m0.750s
sys	0m1.703s

[andrea@andreacrotti tar_baller]$ time ls -lR /home/andrea | cat > /dev/null

real	0m0.986s
user	0m0.413s
sys	0m0.600s


where test_pipe.py is:
from subprocess import PIPE, Popen

# check if doing the pipe with subprocess and with the | is the same or not

pipe_file = open('pipefile', 'w')


p1 = Popen('ls -lR /home/andrea', shell=True, stdout=PIPE, stderr=PIPE)
p2 = Popen('cat', shell=True, stdin=p1.stdout, stdout=PIPE, stderr=PIPE)
p1.stdout.close()

print(p2.stdout.read())


So apparently it's way slower than using this system, is this normal?

Thread

creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-07 17:13 +0000
  Re: creating size-limited tar files Neil Cerutti <neilc@norwich.edu> - 2012-11-07 18:40 +0000
  Re: creating size-limited tar files Alexander Blinne <news@blinne.net> - 2012-11-07 20:05 +0100
    Re: creating size-limited tar files Roy Smith <roy@panix.com> - 2012-11-07 15:32 -0500
      Re: creating size-limited tar files Andrea Crotti <andrea.crotti.0@gmail.com> - 2012-11-07 21:52 +0000
      Re: creating size-limited tar files Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-07 23:15 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-08 10:11 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-08 10:29 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-09 10:39 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-13 10:31 +0000
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-13 09:07 -0700
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-13 09:25 -0700
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-13 09:30 -0700
      Re: creating size-limited tar files Kushal Kumaran <kushal.kumaran+python@gmail.com> - 2012-11-14 11:35 +0530
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-14 00:22 -0700
      Re: creating size-limited tar files Kushal Kumaran <kushal.kumaran+python@gmail.com> - 2012-11-14 14:21 +0530
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-14 11:52 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-14 15:56 +0000
      Re: creating size-limited tar files Dave Angel <d@davea.name> - 2012-11-14 11:10 -0500
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-14 16:16 +0000
  Re: creating size-limited tar files Dave Angel <d@davea.name> - 2012-11-14 11:33 -0500
  Re: creating size-limited tar files Andrea Crotti <andrea.crotti.0@gmail.com> - 2012-11-14 20:43 +0000
  Re: creating size-limited tar files Dave Angel <d@davea.name> - 2012-11-14 15:57 -0500

csiph-web