Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32914

Re: creating size-limited tar files

Path csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <oscar.j.benjamin@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.003
X-Spam-Evidence '*H*': 0.99; '*S*': 0.00; 'argument': 0.04; 'correct.': 0.07; 'implements': 0.07; 'suppose': 0.07; 'chunks': 0.09; 'function:': 0.09; 'impose': 0.09; 'subject:files': 0.09; 'cc:addr:python-list': 0.10; 'files.': 0.13; 'receives': 0.13; 'resulting': 0.13; 'size,': 0.13; 'cases': 0.15; '21:52,': 0.16; 'adjusting': 0.16; 'alexander': 0.16; 'cc:name:python list': 0.16; 'limit.': 0.16; 'roy': 0.16; 'worst': 0.16; 'wrote:': 0.17; '>>>': 0.18; 'define': 0.20; 'file.': 0.20; 'header': 0.24; 'cc:2**1': 0.24; 'pass': 0.25; 'cc:addr:python.org': 0.25; 'header:In-Reply- To:1': 0.25; 'separate': 0.27; 'message-id:@mail.gmail.com': 0.27; 'rest': 0.28; 'post': 0.28; 'cat': 0.29; 'subject:size': 0.29; 'tar': 0.29; 'class': 0.29; 'received:209.85.215.46': 0.30; 'url:python': 0.32; 'file': 0.32; 'suggestion': 0.32; 'could': 0.32; 'right?': 0.33; 'problem': 0.33; 'received:google.com': 0.34; 'acceptable': 0.35; 'pm,': 0.35; 'received:209.85': 0.35; 'something': 0.35; 'next': 0.35; 'but': 0.36; 'url:org': 0.36; 'url:library': 0.36; 'method': 0.36; 'moment': 0.37; 'received:209': 0.37; 'data': 0.37; 'subject:: ': 0.38; 'files': 0.38; 'url:docs': 0.38; 'subject:-': 0.40; 'header:Received:5': 0.40; 'end': 0.40; 'your': 0.60; "you'll": 0.62; 'different': 0.63; 'our': 0.65; 'limit': 0.65; 'biggest': 0.71; 'smith': 0.71; 'article': 0.78; 'andrea': 0.84; 'distributes': 0.84; 'oscar': 0.84; 'write()': 0.91; 'subject:limited': 0.93; 'url:open': 0.96
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=n10JMFTcZdZtkQAaZ7XfDpKS0EuEAwrwKFZxPbJgQTs=; b=jQD89M7lWXOHb9ocTVX6MSKg/3mAPstqUvXY5pLmA5iX0FLjaySsbsiXTYyaxv94kl tWibEapP17N6dTDFI5w9e0iinEO63+jZP8E6ucGGLAQf9WjX3HZ8sTBMYj+/p/DtIxKa 7Qyvfz8+YYKPlqAHLyFwtzRoiIFNFkGXHjT5o//cqvvxxRGBXsamAw6CVecVa2HxFrf6 18tYS32LYW+NKpReInmMdV+rpW67BEkFtO7y8E6IYX69rVv0kSqADZFZQtine2Yi0la7 7/Ka8716LUxU+xf5neKSrzOxWDU+Zm+MBRLsHum2tSg6x6+zqrnQKNbIn+5J7yP4Yc0L tXsg==
MIME-Version 1.0
In-Reply-To <509AD812.2060605@gmail.com>
References <mailman.3381.1352308430.27098.python-list@python.org> <509ab0fa$0$6636$9b4e6d93@newsspool2.arcor-online.net> <roy-2C9B4D.15322307112012@news.panix.com> <509AD812.2060605@gmail.com>
Date Wed, 7 Nov 2012 23:15:14 +0000
Subject Re: creating size-limited tar files
From Oscar Benjamin <oscar.j.benjamin@gmail.com>
To Andrea Crotti <andrea.crotti.0@gmail.com>
Content-Type text/plain; charset=ISO-8859-1
Cc Python List <python-list@python.org>, Roy Smith <roy@panix.com>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.3402.1352330116.27098.python-list@python.org> (permalink)
Lines 35
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1352330116 news.xs4all.nl 6988 [2001:888:2000:d::a6]:35697
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:32914

Show key headers only | View raw

On 7 November 2012 21:52, Andrea Crotti <andrea.crotti.0@gmail.com> wrote:
> On 11/07/2012 08:32 PM, Roy Smith wrote:
>>
>> In article <509ab0fa$0$6636$9b4e6d93@newsspool2.arcor-online.net>,
>>   Alexander Blinne <news@blinne.net> wrote:
>>
>>> I don't know the best way to find the current size, I only have a
>>> general remark.
>>> This solution is not so good if you have to impose a hard limit on the
>>> resulting file size. You could end up having a tar file of size "limit +
>>> size of biggest file - 1 + overhead" in the worst case if the tar is at
>>> limit - 1 and the next file is the biggest file. Of course that may be
>>> acceptable in many cases or it may be acceptable to do something about
>>> it by adjusting the limit.
>
> But the other problem is that at the moment the people that get our chunks
> reassemble the file with a simple:
>
> cat file1.tar.gz file2.tar.gz > file.tar.gz
>
> which I suppose is not going to work if I create 2 different tar files,
> since it would recreate the header in all of the them, right?

Correct. But if you read the rest of Alexander's post you'll find a
suggestion that would work in this case and that can guarantee to give
files of the desired size.

You just need to define your own class that implements a write()
method and then distributes any data it receives to separate files.
You can then pass this as the fileobj argument to the tarfile.open
function:
http://docs.python.org/2/library/tarfile.html#tarfile.open


Oscar

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-07 17:13 +0000
  Re: creating size-limited tar files Neil Cerutti <neilc@norwich.edu> - 2012-11-07 18:40 +0000
  Re: creating size-limited tar files Alexander Blinne <news@blinne.net> - 2012-11-07 20:05 +0100
    Re: creating size-limited tar files Roy Smith <roy@panix.com> - 2012-11-07 15:32 -0500
      Re: creating size-limited tar files Andrea Crotti <andrea.crotti.0@gmail.com> - 2012-11-07 21:52 +0000
      Re: creating size-limited tar files Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-07 23:15 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-08 10:11 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-08 10:29 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-09 10:39 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-13 10:31 +0000
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-13 09:07 -0700
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-13 09:25 -0700
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-13 09:30 -0700
      Re: creating size-limited tar files Kushal Kumaran <kushal.kumaran+python@gmail.com> - 2012-11-14 11:35 +0530
      Re: creating size-limited tar files Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-14 00:22 -0700
      Re: creating size-limited tar files Kushal Kumaran <kushal.kumaran+python@gmail.com> - 2012-11-14 14:21 +0530
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-14 11:52 +0000
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-14 15:56 +0000
      Re: creating size-limited tar files Dave Angel <d@davea.name> - 2012-11-14 11:10 -0500
      Re: creating size-limited tar files andrea crotti <andrea.crotti.0@gmail.com> - 2012-11-14 16:16 +0000
  Re: creating size-limited tar files Dave Angel <d@davea.name> - 2012-11-14 11:33 -0500
  Re: creating size-limited tar files Andrea Crotti <andrea.crotti.0@gmail.com> - 2012-11-14 20:43 +0000
  Re: creating size-limited tar files Dave Angel <d@davea.name> - 2012-11-14 15:57 -0500

csiph-web