Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <andrea.crotti.0@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.023
X-Spam-Evidence: '*H*': 0.95; '*S*': 0.00; 'received:209.85.223': 0.03; 'chunks': 0.09; 'mode,': 0.09; 'subject:files': 0.09; 'to:name:python-list': 0.15; 'compress': 0.16; 'simplest': 0.16; 'tarfile': 0.16; 'replacing': 0.17; '(in': 0.18; 'script': 0.24; 'skip:" 20': 0.26; 'message-id:@mail.gmail.com': 0.27; "doesn't": 0.28; 'actual': 0.28; 'subject:size': 0.29; 'tar': 0.29; "i'm": 0.29; 'file': 0.32; 'to:addr:python-list': 0.33; 'received:google.com': 0.34; 'thanks': 0.34; 'exist': 0.35; 'problem,': 0.35; 'doing': 0.35; 'received:209.85': 0.35; 'there': 0.35; 'but': 0.36; 'should': 0.36; 'moment': 0.37; 'received:209': 0.37; 'data': 0.37; 'to:addr:python.org': 0.39; 'subject:-': 0.40; 'header:Received:5': 0.40; 'skip:n 10': 0.63; 'maybe,': 0.84; 'size?': 0.84; 'subject:limited': 0.93
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=kDkCVWROWvjQry8OxmGIAh/qVl2EXi/8KG9aC8M2UFc=; b=snwGHpQO7NEfiNmyihdptt3ojEd65yPtRWLNAhcUoLNOXPYA9WlrCRNvBx5AZuAIYz mPnfjPTCqYKv7LCyttlSR2JyhFZVaNvS1F85SlbPc4YXh+wKfyPQ2R9vlpD/uAW/0Q9w DWTSFfH5tVnfEfhsfMHE3CdLwxELtt+jCmAeFJVjmM2ZTSHC7pnBvoPLEm2icYBRIr4t pb3jOmvTfHNzDzwLhYxDFxGb4vIfZ7ZwHQDFUwrRywQ8v4h7uHTwJS7zALSXbDmU+Umj BBEu+Wt4vENzLEfd97WFqhWxSJsmjabXJ0Pq+ZRhaK6wLgGQ8iJ10h8q3E/bWYydJtb4 a4Ww==
MIME-Version: 1.0
Date: Wed, 7 Nov 2012 17:13:47 +0000
Subject: creating size-limited tar files
From: andrea crotti <andrea.crotti.0@gmail.com>
To: python-list <python-list@python.org>
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.3381.1352308430.27098.python-list@python.org>
Lines: 24
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1352308430 news.xs4all.nl 6970 [2001:888:2000:d::a6]:38167
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:32896

Simple problem, given a lot of data in many files/directories, I
should create a tar file splitted in chunks <= a given size.

The simplest way would be to compress the whole thing and then split.

At the moment the actual script which I'm replacing is doing a
"system('split..')", which is not that great, so I would like to do it
while compressing.

So I thought about (in pseudocode)


while remaining_files:
    tar_file.addfile(remaining_files.pop())
    if size(tar_file) >= limit:
         close(tar_file)
         tar_file = new_tar_file()

which might work maybe, but how do I get the current size?  There
should be tarinfo.size but it doesn't exist on a TarFile opened in
write mode, so should I do a stat after each flush?

Any other better ideas otherwise?
thanks