Re: best way to handle this in Python

References	<CAOF-KfjZN8Tks+Nex=E9vrFqJN_t+j8hpkhQV=BGzEtno8b0Yg@mail.gmail.com> <5008ABD5.8020407@davea.name> <CAOF-KfgUdEGZRdynU81HzRFzVuf8JfX8WkSsV6xVmXTamObukA@mail.gmail.com> <6emh0859cren5ond0k5n2f58mh36bnp9jc@invalid.netcom.com> <CAOF-KfgOkB5eWYO4y8HiPcNS1bBh5SKcpiX1Ctz6K9GRdgYgUw@mail.gmail.com>
From	Ian Kelly <ian.g.kelly@gmail.com>
Date	2012-07-20 12:14 -0600
Subject	Re: best way to handle this in Python
Newsgroups	comp.lang.python
Message-ID	<mailman.2351.1342808103.4697.python-list@python.org> (permalink)

Show all headers | View raw

On Fri, Jul 20, 2012 at 4:34 AM, Rita <rmorgan466@gmail.com> wrote:
> Thats an interesting data structure Dennis. I will actually be running this
> type of query many times preferable in an ad-hoc environment. That makes it
> tough for sqlite3 since there will be several hundred thousand tuples.

Several hundred thousand is not an enormous number.  I think you're
underestimating sqlite3.  I just tried a test with one million tuples,
six colors per tuple (six million rows altogether).  Each row contains
a primary key, a timestamp, a color, and a count, with an index on the
timestamp column.  Building the database from scratch took about a
minute; adding the index took about another minute.  Incremental
updates would of course be much faster.  Queries like "select * from
data where timestamp between 500000 and 600000" return instantly (from
a user perspective).

Cheers,
Ian

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: best way to handle this in Python Ian Kelly <ian.g.kelly@gmail.com> - 2012-07-20 12:14 -0600
  Re: best way to handle this in Python Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-07-21 00:43 +0000

csiph-web