Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32310

Re: Fast forward-backward (write-read)

Show key headers only | View raw

On 10/24/2012 03:14 AM, Virgil Stokes wrote:
> On 24-Oct-2012 01:46, Paul Rubin wrote:
>> Virgil Stokes <vs@it.uu.se> writes:
>>> Yes, I do wish to inverse the order,  but the "forward in time" file
>>> will be in binary.
>> I really think it will be simplest to just write the file in forward
>> order, then use mmap to read it one record at a time.  It might be
>> possible to squeeze out a little more performance with reordering tricks
>> but that's the first thing to try.
> Thanks Paul,
> I am working on this approach now...

If you're using mmap to map the whole file, you'll need 64bit Windows to
start with.  I'd be interested to know if Windows will allow you to mmap
100gb at one stroke.  Have you tried it, or are you starting by figuring
how to access the data from the mmap?

-- 

DaveA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: Fast forward-backward (write-read) Virgil Stokes <vs@it.uu.se> - 2012-10-23 20:37 +0200
  Re: Fast forward-backward (write-read) Paul Rubin <no.email@nospam.invalid> - 2012-10-23 16:46 -0700
    Re: Fast forward-backward (write-read) Dave Angel <d@davea.name> - 2012-10-28 07:18 -0400
    Re: Fast forward-backward (write-read) Virgil Stokes <vs@it.uu.se> - 2012-10-28 15:20 +0100
    Re: Fast forward-backward (write-read) Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-10-28 18:21 +0000
    Re: Fast forward-backward (write-read) Virgil Stokes <vs@it.uu.se> - 2012-10-28 23:36 +0100

csiph-web