Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #65001

Re: fseek In Compressed Files

Show key headers only | View raw

On Fri, Jan 31, 2014 at 12:34 AM, Ayushi Dalmia
<ayushidalmia2604@gmail.com> wrote:
> where temp.txt is the posting list file which is first written in a compressed format and then read  later.

Unless you specify otherwise, a compressed file is likely to have
sub-byte boundaries. It might not be possible to seek to a specific
line.

What you could do, though, is explicitly compress each line, then
write out separately-compressed blocks. You can then seek to any one
that you want, read it, and decompress it. But at this point, you're
probably going to do better with a database; PostgreSQL, for instance,
will automatically compress any content that it believes it's
worthwhile to compress (as long as it's in a VARCHAR field or similar
and the table hasn't been configured to prevent that, yada yada). All
you have to do is tell Postgres to store this, retrieve that, and
it'll worry about the details of compression and decompression. As an
added benefit, you can divide the text up and let it do the hard work
of indexing, filtering, sorting, etc. I suspect you'll find that
deploying a database is a much more efficient use of your development
time than recreating all of that.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

fseek In Compressed Files Ayushi Dalmia <ayushidalmia2604@gmail.com> - 2014-01-30 02:50 -0800
  Re: fseek In Compressed Files Peter Otten <__peter__@web.de> - 2014-01-30 12:28 +0100
  Re:fseek In Compressed Files Dave Angel <davea@davea.name> - 2014-01-30 06:55 -0500
  Re: fseek In Compressed Files Ayushi Dalmia <ayushidalmia2604@gmail.com> - 2014-01-30 05:34 -0800
    Re: fseek In Compressed Files Chris Angelico <rosuav@gmail.com> - 2014-01-31 00:49 +1100
    Re: fseek In Compressed Files Dave Angel <davea@davea.name> - 2014-02-03 15:57 -0500
      Re: fseek In Compressed Files Ayushi Dalmia <ayushidalmia2604@gmail.com> - 2014-02-04 04:39 -0800
  Re: fseek In Compressed Files Serhiy Storchaka <storchaka@gmail.com> - 2014-01-30 17:02 +0200
  Re: fseek In Compressed Files Ayushi Dalmia <ayushidalmia2604@gmail.com> - 2014-01-30 07:37 -0800
    Re: fseek In Compressed Files Dave Angel <davea@davea.name> - 2014-01-30 13:46 -0500
      Re: fseek In Compressed Files Ayushi Dalmia <ayushidalmia2604@gmail.com> - 2014-01-31 21:52 -0800
        Re: fseek In Compressed Files Dave Angel <davea@davea.name> - 2014-02-01 04:38 -0500
  Re: fseek In Compressed Files Peter Otten <__peter__@web.de> - 2014-01-30 17:21 +0100
    Re: fseek In Compressed Files Ayushi Dalmia <ayushidalmia2604@gmail.com> - 2014-01-31 21:50 -0800
  Re: fseek In Compressed Files Serhiy Storchaka <storchaka@gmail.com> - 2014-02-03 20:32 +0200

csiph-web