Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.002 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'python,': 0.02; 'broken': 0.03; 'argument': 0.04; 'correct.': 0.07; 'implements': 0.07; 'skip:/ 10': 0.07; 'filesystem': 0.09; 'function:': 0.09; 'subject:files': 0.09; 'cc:addr:python-list': 0.10; 'files.': 0.13; 'receives': 0.13; '(the': 0.15; 'benjamin': 0.16; 'cc:name:python list': 0.16; 'reliably': 0.16; 'define': 0.20; 'split': 0.23; 'this:': 0.23; 'cc:2**1': 0.24; 'linux': 0.24; 'script': 0.24; 'pass': 0.25; 'cc:addr:python.org': 0.25; 'header :In-Reply-To:1': 0.25; 'separate': 0.27; 'message- id:@mail.gmail.com': 0.27; 'rest': 0.28; 'run': 0.28; 'post': 0.28; 'subject:size': 0.29; 'tar': 0.29; 'probably': 0.29; 'class': 0.29; 'received:209.85.210.174': 0.30; 'url:python': 0.32; 'suggestion': 0.32; 'doubt': 0.33; 'received:google.com': 0.34; 'list': 0.35; 'fail': 0.35; 'doing': 0.35; 'received:209.85': 0.35; 'but': 0.36; 'url:org': 0.36; 'url:library': 0.36; 'method': 0.36; 'should': 0.36; 'bad': 0.37; 'received:209': 0.37; 'data': 0.37; 'subject:: ': 0.38; 'files': 0.38; 'some': 0.38; 'url:docs': 0.38; 'subject:-': 0.40; 'header:Received:5': 0.40; 'your': 0.60; "you'll": 0.62; 'more': 0.63; 'saw': 0.75; 'distributes': 0.84; 'easier,': 0.84; 'oscar': 0.84; 'write()': 0.91; 'subject:limited': 0.93; 'url:open': 0.96 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=h6Mn+Nbl8jOM5sgn1+lXOFbeNfJHzJSxye8nWSHOTbU=; b=syuD4fonhhvV8dJnlzNnTEa7756gZn06Gqp0XKBG7W+rqEZGHoZ9W5IDzl5360AOzN lsTIbI1685ZIUy59hudu5Q1gQx6IifarEg+TuXR4REAfkDX76ndUEFAjKCj9Mj5kLUKC qTM5RUACaA/9BURgbHcg91H+JPWXrIeNAXH8d6LA4Jy/jqnQrZHe15ugs9LfkRintUnq +T1jp0xs7jcYdJBbpHjReTQ2e4NzS+HmvsFec5lUXWjeNiWJNdC3yBGPse0X5m4NcPSw JjYg/4Pn9K3q7F26rfvnhvh9s8TzmC8kUlbqvZMyGuthKDy676FNu4HuhrVNYZ2tgnLO lffg== MIME-Version: 1.0 In-Reply-To: References: <509ab0fa$0$6636$9b4e6d93@newsspool2.arcor-online.net> <509AD812.2060605@gmail.com> Date: Thu, 8 Nov 2012 10:11:44 +0000 Subject: Re: creating size-limited tar files From: andrea crotti To: Oscar Benjamin Content-Type: text/plain; charset=ISO-8859-1 Cc: Python List , Roy Smith X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 27 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1352369507 news.xs4all.nl 6877 [2001:888:2000:d::a6]:42029 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:32938 2012/11/7 Oscar Benjamin : > > Correct. But if you read the rest of Alexander's post you'll find a > suggestion that would work in this case and that can guarantee to give > files of the desired size. > > You just need to define your own class that implements a write() > method and then distributes any data it receives to separate files. > You can then pass this as the fileobj argument to the tarfile.open > function: > http://docs.python.org/2/library/tarfile.html#tarfile.open > > > Oscar Yes yes I saw the answer, but now I was thinking that what I need is simply this: tar czpvf - /path/to/archive | split -d -b 100M - tardisk since it should run only on Linux it's probably way easier, my script will then only need to create the list of files to tar.. The only doubt is if this is more or less reliably then doing it in Python, when can this fail with some bad broken pipe? (the filesystem is not very good as I said and it's mounted with NFS)