Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #102793 > unrolled thread

tarfile : read from a socket?

Started byUlli Horlacher <framstag@rus.uni-stuttgart.de>
First post2016-02-11 08:31 +0000
Last post2016-02-12 12:48 +0000
Articles 20 — 10 participants

Back to article view | Back to comp.lang.python


Contents

  tarfile : read from a socket? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-11 08:31 +0000
    Re: tarfile : read from a socket? INADA Naoki <songofacandy@gmail.com> - 2016-02-11 19:26 +0900
    Re: tarfile : read from a socket? Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-02-11 11:47 +0100
      Re: tarfile : read from a socket? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-11 12:53 +0000
        Re: tarfile : read from a socket? MRAB <python@mrabarnett.plus.com> - 2016-02-11 13:16 +0000
        Re: tarfile : read from a socket? Chris Angelico <rosuav@gmail.com> - 2016-02-12 00:18 +1100
          Re: tarfile : read from a socket? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-11 13:34 +0000
        Re: tarfile : read from a socket? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-11 13:25 +0000
          Re: tarfile : read from a socket? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-11 16:41 +0000
            Re: tarfile : read from a socket? MRAB <python@mrabarnett.plus.com> - 2016-02-11 17:10 +0000
            modifying a standard module? (was: Re: tarfile : read from a socket?) Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-11 17:10 +0000
              Re: modifying a standard module? (was: Re: tarfile : read from a socket?) Matt Wheeler <m@funkyhat.org> - 2016-02-12 20:53 +0000
                Re: modifying a standard module? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-12 22:46 +0000
                  Re: modifying a standard module? Paul Rubin <no.email@nospam.invalid> - 2016-02-12 16:21 -0800
            Re: tarfile : read from a socket? Peter Otten <__peter__@web.de> - 2016-02-11 18:18 +0100
            Re: tarfile : read from a socket? Lars Gustäbel <lars@gustaebel.de> - 2016-02-11 18:27 +0100
            Re: tarfile : read from a socket? Random832 <random832@fastmail.com> - 2016-02-11 16:23 -0500
            Re: tarfile : read from a socket? Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2016-02-12 09:35 +0100
            Re: tarfile : read from a socket? Lars Gustäbel <lars@gustaebel.de> - 2016-02-12 09:40 +0100
              Re: tarfile : read from a socket? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2016-02-12 12:48 +0000

#102793 — tarfile : read from a socket?

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-11 08:31 +0000
Subjecttarfile : read from a socket?
Message-ID<n9hgtb$rls$1@news2.informatik.uni-stuttgart.de>
https://docs.python.org/2/library/tarfile.html says:

 tarfile.open(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs)

    Return a TarFile object for the pathname name.


(How) can I read a tar file from a (tcp) socket?
I do not have a pathname but a socket object from socket.create_connection() 

    

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [next] | [standalone]


#102798

FromINADA Naoki <songofacandy@gmail.com>
Date2016-02-11 19:26 +0900
Message-ID<mailman.41.1455186416.22075.python-list@python.org>
In reply to#102793
Have you tried socket.makefile() method?

[toc] | [prev] | [next] | [standalone]


#102800

FromAntoon Pardon <antoon.pardon@rece.vub.ac.be>
Date2016-02-11 11:47 +0100
Message-ID<mailman.43.1455187713.22075.python-list@python.org>
In reply to#102793
On 02/11/2016 09:31 AM, Ulli Horlacher wrote:
> https://docs.python.org/2/library/tarfile.html says:
>
>  tarfile.open(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs)
>
>     Return a TarFile object for the pathname name.
>
>
> (How) can I read a tar file from a (tcp) socket?
> I do not have a pathname but a socket object from socket.create_connection

# First you construct a file object with makefile.

fo = socket.makefile()

# Then you use the fileobj argument with tarfile.open.

tarfile.open(mode='r', fileobj = fo)

[toc] | [prev] | [next] | [standalone]


#102804

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-11 12:53 +0000
Message-ID<n9i07l$vgl$1@news2.informatik.uni-stuttgart.de>
In reply to#102800
Antoon Pardon <antoon.pardon@rece.vub.ac.be> wrote:

> > (How) can I read a tar file from a (tcp) socket?
> > I do not have a pathname but a socket object from socket.create_connection
> 
> # First you construct a file object with makefile.
> 
> fo = socket.makefile()
> 
> # Then you use the fileobj argument with tarfile.open.
> 
> tarfile.open(mode='r', fileobj = fo)


I have:

  sock = socket.create_connection((server,port))
  bs = kB64
  taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
  
  
        
Traceback (most recent call last):
  (...)
  File "./fexit.py", line 1838, in sex_send
    taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
  File "/usr/lib/python2.7/tarfile.py", line 1695, in open
    return cls.taropen(name, mode, fileobj, **kwargs)
  File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen
    return cls(name, mode, fileobj, **kwargs)
  File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__
    self.offset = self.fileobj.tell()
AttributeError: '_fileobject' object has no attribute 'tell'

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]


#102805

FromMRAB <python@mrabarnett.plus.com>
Date2016-02-11 13:16 +0000
Message-ID<mailman.47.1455196622.22075.python-list@python.org>
In reply to#102804
On 2016-02-11 12:53, Ulli Horlacher wrote:
> Antoon Pardon <antoon.pardon@rece.vub.ac.be> wrote:
>
>> > (How) can I read a tar file from a (tcp) socket?
>> > I do not have a pathname but a socket object from socket.create_connection
>>
>> # First you construct a file object with makefile.
>>
>> fo = socket.makefile()
>>
>> # Then you use the fileobj argument with tarfile.open.
>>
>> tarfile.open(mode='r', fileobj = fo)
>
>
> I have:
>
>    sock = socket.create_connection((server,port))
>    bs = kB64
>    taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
>
>
>
> Traceback (most recent call last):
>    (...)
>    File "./fexit.py", line 1838, in sex_send
>      taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
>    File "/usr/lib/python2.7/tarfile.py", line 1695, in open
>      return cls.taropen(name, mode, fileobj, **kwargs)
>    File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen
>      return cls(name, mode, fileobj, **kwargs)
>    File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__
>      self.offset = self.fileobj.tell()
> AttributeError: '_fileobject' object has no attribute 'tell'
>
I suppose you could write your own class to wrap the socket and provide 
the required methods.

[toc] | [prev] | [next] | [standalone]


#102806

FromChris Angelico <rosuav@gmail.com>
Date2016-02-12 00:18 +1100
Message-ID<mailman.48.1455196694.22075.python-list@python.org>
In reply to#102804
On Thu, Feb 11, 2016 at 11:53 PM, Ulli Horlacher
<framstag@rus.uni-stuttgart.de> wrote:
> I have:
>
>   sock = socket.create_connection((server,port))
>   bs = kB64
>   taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
>
>
>
> Traceback (most recent call last):
>   (...)
>   File "./fexit.py", line 1838, in sex_send
>     taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
>   File "/usr/lib/python2.7/tarfile.py", line 1695, in open
>     return cls.taropen(name, mode, fileobj, **kwargs)
>   File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen
>     return cls(name, mode, fileobj, **kwargs)
>   File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__
>     self.offset = self.fileobj.tell()
> AttributeError: '_fileobject' object has no attribute 'tell'

Sounds like tarfile needs a seekable file. How big is this file you're
reading? Can you simply read the whole thing into memory, then use
io.BytesIO? I had a quick glance at help(BytesIO) but didn't find a
simple way to make a buffer that reads from an upstream file when it
needs more content, but it should be possible to build one.

ChrisA

[toc] | [prev] | [next] | [standalone]


#102809

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-11 13:34 +0000
Message-ID<n9i2ll$vva$2@news2.informatik.uni-stuttgart.de>
In reply to#102806
Chris Angelico <rosuav@gmail.com> wrote:

> Sounds like tarfile needs a seekable file. How big is this file you're
> reading?

No limits. It can be many TBs...

The use case is:

http://fex.rus.uni-stuttgart.de:8080/


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]


#102808

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-11 13:25 +0000
Message-ID<n9i246$vva$1@news2.informatik.uni-stuttgart.de>
In reply to#102804
Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:

> I have:
> 
>   sock = socket.create_connection((server,port))
>   bs = kB64
>   taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
>   
>   
>         
> Traceback (most recent call last):
>   (...)
>   File "./fexit.py", line 1838, in sex_send
>     taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w')
>   File "/usr/lib/python2.7/tarfile.py", line 1695, in open
>     return cls.taropen(name, mode, fileobj, **kwargs)
>   File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen
>     return cls(name, mode, fileobj, **kwargs)
>   File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__
>     self.offset = self.fileobj.tell()
> AttributeError: '_fileobject' object has no attribute 'tell'

Reading the doc helps :-)

https://docs.python.org/2/library/tarfile.html

  For special purposes, there is a second format for mode:
  'filemode|[compression]'. tarfile.open() will return a TarFile object
  that processes its data as a stream of blocks.

With 

  taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|')

I get no more error.


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]


#102814

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-11 16:41 +0000
Message-ID<n9idk7$37f$1@news2.informatik.uni-stuttgart.de>
In reply to#102808
Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:

> With 
> 
>   taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|')
> 
> I get no more error.

Of course, this is the writing client.

Now I have a small problem with the reading client.

This code works so far:

  sfo = sock.makefile('r')
  taro = tarfile.open(fileobj=sfo,mode='r|')
  taro.extractall(path=edir)

But it does not writes anything to the terminal to inform the user.

When I use:

  for member in taro.getmembers():
      print('extracting "%s"' % member.name)
      taro.extract(member)

I get the error:

  File "/usr/lib/python2.7/tarfile.py", line 556, in seek
    raise StreamError("seeking backwards is not allowed")

Of course, a stream is not seekable.

Any ideas?

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]


#102818

FromMRAB <python@mrabarnett.plus.com>
Date2016-02-11 17:10 +0000
Message-ID<mailman.54.1455210623.22075.python-list@python.org>
In reply to#102814
On 2016-02-11 16:41, Ulli Horlacher wrote:
> Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:
>
>> With
>>
>>   taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|')
>>
>> I get no more error.
>
> Of course, this is the writing client.
>
> Now I have a small problem with the reading client.
>
> This code works so far:
>
>    sfo = sock.makefile('r')
>    taro = tarfile.open(fileobj=sfo,mode='r|')
>    taro.extractall(path=edir)
>
> But it does not writes anything to the terminal to inform the user.
>
> When I use:
>
>    for member in taro.getmembers():
>        print('extracting "%s"' % member.name)
>        taro.extract(member)
>
> I get the error:
>
>    File "/usr/lib/python2.7/tarfile.py", line 556, in seek
>      raise StreamError("seeking backwards is not allowed")
>
> Of course, a stream is not seekable.
>
> Any ideas?
>
Try this:

member = taro.next()
while member is not None:
     print('extracting "%s"' % member.name)
     taro.extract(member)
     member = tar.next()

[toc] | [prev] | [next] | [standalone]


#102819 — modifying a standard module? (was: Re: tarfile : read from a socket?)

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-11 17:10 +0000
Subjectmodifying a standard module? (was: Re: tarfile : read from a socket?)
Message-ID<n9ifak$3l7$1@news2.informatik.uni-stuttgart.de>
In reply to#102814
Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:

> This code works so far:
> 
>   sfo = sock.makefile('r')
>   taro = tarfile.open(fileobj=sfo,mode='r|')
>   taro.extractall(path=edir)
> 
> But it does not writes anything to the terminal to inform the user.
> 
> When I use:
> 
>   for member in taro.getmembers():
>       print('extracting "%s"' % member.name)
>       taro.extract(member)
> 
> I get the error:
> 
>   File "/usr/lib/python2.7/tarfile.py", line 556, in seek
>     raise StreamError("seeking backwards is not allowed")
> 
> Of course, a stream is not seekable.
> 
> Any ideas?

As a hack, I modified the standard library module tarfile.py:

root@diaspora:/usr/lib/python2.7# vv -d
--- ./.versions/tarfile.py~1~   2015-06-22 21:59:27.000000000 +0200
+++ tarfile.py  2016-02-11 18:01:50.185555952 +0100
@@ -2045,6 +2045,7 @@
                 directories.append(tarinfo)
                 tarinfo = copy.copy(tarinfo)
                 tarinfo.mode = 0700
+            print('untar "%s"' % tarinfo.name)
             self.extract(tarinfo, path)
 
         # Reverse sort directories.


This gives me exact the output I want :-)

BUT I want to distribute my program and all others will not see the tar
extracting information.

Now my question:

How can I substitute the standard module function tarfile.extractall() with
my own function?



-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]


#102871 — Re: modifying a standard module? (was: Re: tarfile : read from a socket?)

FromMatt Wheeler <m@funkyhat.org>
Date2016-02-12 20:53 +0000
SubjectRe: modifying a standard module? (was: Re: tarfile : read from a socket?)
Message-ID<mailman.85.1455310425.22075.python-list@python.org>
In reply to#102819
On 11 February 2016 at 17:10, Ulli Horlacher
<framstag@rus.uni-stuttgart.de> wrote:
>
> Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:
> As a hack, I modified the standard library module tarfile.py:
>
> root@diaspora:/usr/lib/python2.7# vv -d
> --- ./.versions/tarfile.py~1~   2015-06-22 21:59:27.000000000 +0200
> +++ tarfile.py  2016-02-11 18:01:50.185555952 +0100
> @@ -2045,6 +2045,7 @@
>                  directories.append(tarinfo)
>                  tarinfo = copy.copy(tarinfo)
>                  tarinfo.mode = 0700
> +            print('untar "%s"' % tarinfo.name)
>              self.extract(tarinfo, path)
>
>          # Reverse sort directories.
>
>
> This gives me exact the output I want :-)
>
> BUT I want to distribute my program and all others will not see the tar
> extracting information.
>
> Now my question:
>
> How can I substitute the standard module function tarfile.extractall() with
> my own function?

import tarfile
def new_extractall(self, *args, **kwargs):
    print("I am a function. Woohoo!")

tarfile.TarFile.extractall = new_extractall

But bear in mind that that will change tarfile.extractall for every
single module that imports it within the same python process. Is that
really what you want?


Is there a reason you can't subclass TarFile as others have suggested?

Perhaps even this is enough:

class NoisyTarFile(TarFile):
    """untested, sorry"""
    def extract(self, member, *args, **kwargs):
        print('extracting "%s"' % member.name)
        super(NoisyTarFile, self).extract(member, *args, **kwargs)

As the very next step after your print in extractall is a call to
extract anyway?


If you must patch the standard library tarfile module then I would
suggest patching it to have an extra, default False, argument to
enable your printing behaviour, so you don't risk messing up anyone
else's use of it.


-- 
Matt Wheeler
http://funkyh.at

[toc] | [prev] | [next] | [standalone]


#102875 — Re: modifying a standard module?

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-12 22:46 +0000
SubjectRe: modifying a standard module?
Message-ID<n9lnd3$u3c$1@news2.informatik.uni-stuttgart.de>
In reply to#102871
Matt Wheeler <m@funkyhat.org> wrote:

> > How can I substitute the standard module function tarfile.extractall() with
> > my own function?
> 
> import tarfile
> def new_extractall(self, *args, **kwargs):
>     print("I am a function. Woohoo!")
> 
> tarfile.TarFile.extractall = new_extractall

This is more easy than I could imagined :-)
It is in my Python notes, now.


> But bear in mind that that will change tarfile.extractall for every
> single module that imports it within the same python process. Is that
> really what you want?

Yes. I have no own modules. Just one program file.


> Is there a reason you can't subclass TarFile as others have suggested?

The reason: I have no ideas on classes :-}

Of course, I should have to learn about, but until now it was not necessary.
The other solutions in this thread are sufficent for me.

Meanwhile I have implemented the iterator function:

    taro.extractall(members=itar(taro),path=edir)

def itar(tar):
  for ti in tar:
    # minimal protection against dangerous file names
    # see http://bugs.python.org/issue21109#msg215656
    ti.name = subst(r'^(?i)([a-z]:)?(\.\.)?[/\\]','',ti.name)
    print('untar "%s"' % ti.name)
    yield ti
    

Perfekt solution for me :-)
Thanks to all.


> If you must patch the standard library tarfile module then I would
> suggest patching it to have an extra, default False, argument to
> enable your printing behaviour, so you don't risk messing up anyone
> else's use of it.

Yes, a good idea.


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]


#102877 — Re: modifying a standard module?

FromPaul Rubin <no.email@nospam.invalid>
Date2016-02-12 16:21 -0800
SubjectRe: modifying a standard module?
Message-ID<87bn7lcved.fsf@nightsong.com>
In reply to#102875
Ulli Horlacher <framstag@rus.uni-stuttgart.de> writes:
>> tarfile.TarFile.extractall = new_extractall
>
> This is more easy than I could imagined :-) It is in my Python notes,
> now.

This is called "duck punching" or "monkey patching" and sometimes it's
necessary, but it's something of an antipattern since the module could
change under you between versions, and that sort of thing.  If you have
to do it, then fine; but if you have a choice, it's preferable to avoid
that sort of thing.

[toc] | [prev] | [next] | [standalone]


#102820

FromPeter Otten <__peter__@web.de>
Date2016-02-11 18:18 +0100
Message-ID<mailman.55.1455211152.22075.python-list@python.org>
In reply to#102814
Ulli Horlacher wrote:

> Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:
> 
>> With
>> 
>>   taro = tarfile.open(fileobj=sock.makefile('w',kB64),mode='w|')
>> 
>> I get no more error.
> 
> Of course, this is the writing client.
> 
> Now I have a small problem with the reading client.
> 
> This code works so far:
> 
>   sfo = sock.makefile('r')
>   taro = tarfile.open(fileobj=sfo,mode='r|')
>   taro.extractall(path=edir)
> 
> But it does not writes anything to the terminal to inform the user.
> 
> When I use:
> 
>   for member in taro.getmembers():
>       print('extracting "%s"' % member.name)
>       taro.extract(member)
> 
> I get the error:
> 
>   File "/usr/lib/python2.7/tarfile.py", line 556, in seek
>     raise StreamError("seeking backwards is not allowed")
> 
> Of course, a stream is not seekable.
> 
> Any ideas?

A look into the source is often helpful ;)

$ cat extract_from_stream.py
import sys
from tarfile import TarFile

class MyTarFile(TarFile):
    def extract(self, member, path="."):
        print "extracting", member
        return TarFile.extract(self, member, path)

tf = MyTarFile.open(fileobj=sys.stdin, mode="r|")
tf.extractall()
$ touch foo bar
$ tar -cf archive.tar foo bar
$ python extract_from_stream.py < archive.tar 
extracting <TarInfo 'foo' at 0x7f2b3f394890>
extracting <TarInfo 'bar' at 0x7f2b3f3a4bd0>

[toc] | [prev] | [next] | [standalone]


#102821

FromLars Gustäbel <lars@gustaebel.de>
Date2016-02-11 18:27 +0100
Message-ID<mailman.56.1455211833.22075.python-list@python.org>
In reply to#102814
On Thu, Feb 11, 2016 at 04:41:43PM +0000, Ulli Horlacher wrote:
>   sfo = sock.makefile('r')
>   taro = tarfile.open(fileobj=sfo,mode='r|')
>   taro.extractall(path=edir)

What about using an iterator?

def myiter(tar):
    for t in tar:
        print "extracting", t.name
        yield t

sfo = sock.makefile('r')
taro = tarfile.open(fileobj=sfo,mode='r|')
taro.extractall(members=myiter(taro),path=edir)

Cheers,

-- 
Lars Gustäbel
lars@gustaebel.de

[toc] | [prev] | [next] | [standalone]


#102831

FromRandom832 <random832@fastmail.com>
Date2016-02-11 16:23 -0500
Message-ID<mailman.59.1455225841.22075.python-list@python.org>
In reply to#102814
On Thu, Feb 11, 2016, at 11:41, Ulli Horlacher wrote:
> When I use:
> 
>   for member in taro.getmembers():
>       print('extracting "%s"' % member.name)
>       taro.extract(member)
> 
> I get the error:
> 
>   File "/usr/lib/python2.7/tarfile.py", line 556, in seek
>     raise StreamError("seeking backwards is not allowed")
> 
> Of course, a stream is not seekable.
> 
> Any ideas?

Try this:

while True:
    member = taro.next()
    if member is None: break
    print('extracting "%s"' % member.name)
    taro.extract(member)

[toc] | [prev] | [next] | [standalone]


#102855

FromAntoon Pardon <antoon.pardon@rece.vub.ac.be>
Date2016-02-12 09:35 +0100
Message-ID<mailman.73.1455266145.22075.python-list@python.org>
In reply to#102814
On 02/11/2016 06:27 PM, Lars Gustäbel wrote:
> On Thu, Feb 11, 2016 at 04:41:43PM +0000, Ulli Horlacher wrote:
>>   sfo = sock.makefile('r')
>>   taro = tarfile.open(fileobj=sfo,mode='r|')
>>   taro.extractall(path=edir)
> What about using an iterator?
>
> def myiter(tar):
>     for t in tar:
>         print "extracting", t.name
>         yield t
>
> sfo = sock.makefile('r')
> taro = tarfile.open(fileobj=sfo,mode='r|')
> taro.extractall(members=myiter(taro),path=edir)
>
> Cheers,

The tarfile is already an iterator. Just do the following: for ti in
taro:     print "extracting", ti.name taro.extract(ti)

[toc] | [prev] | [next] | [standalone]


#102856

FromLars Gustäbel <lars@gustaebel.de>
Date2016-02-12 09:40 +0100
Message-ID<mailman.74.1455266442.22075.python-list@python.org>
In reply to#102814
On Fri, Feb 12, 2016 at 09:35:40AM +0100, Antoon Pardon wrote:
> On 02/11/2016 06:27 PM, Lars Gustäbel wrote:
> > What about using an iterator?
> >
> > def myiter(tar):
> >     for t in tar:
> >         print "extracting", t.name
> >         yield t
> >
> > sfo = sock.makefile('r')
> > taro = tarfile.open(fileobj=sfo,mode='r|')
> > taro.extractall(members=myiter(taro),path=edir)
> >
> The tarfile is already an iterator. Just do the following: for ti in
> taro:     print "extracting", ti.name taro.extract(ti)

The extractall() method does a little bit more than just extract(), i.e.
setting directory mtimes, see
https://docs.python.org/3.5/library/tarfile.html#tarfile.TarFile.extractall

-- 
Lars Gustäbel
lars@gustaebel.de

[toc] | [prev] | [next] | [standalone]


#102860

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2016-02-12 12:48 +0000
Message-ID<n9kkau$l9d$1@news2.informatik.uni-stuttgart.de>
In reply to#102856
Lars Gustäbel <lars@gustaebel.de> wrote:
> On Fri, Feb 12, 2016 at 09:35:40AM +0100, Antoon Pardon wrote:
> > On 02/11/2016 06:27 PM, Lars Gustäbel wrote:
> > > What about using an iterator?
> > >
> > > def myiter(tar):
> > >     for t in tar:
> > >         print "extracting", t.name
> > >         yield t
> > >
> > > sfo = sock.makefile('r')
> > > taro = tarfile.open(fileobj=sfo,mode='r|')
> > > taro.extractall(members=myiter(taro),path=edir)
> > >
> > The tarfile is already an iterator. Just do the following: for ti in
> > taro:     print "extracting", ti.name taro.extract(ti)
> 
> The extractall() method does a little bit more than just extract(), i.e.
> setting directory mtimes, see
> https://docs.python.org/3.5/library/tarfile.html#tarfile.TarFile.extractall

This is an important hint!
Thanks!


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [standalone]


Back to top | Article view | comp.lang.python


csiph-web