Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #25700

Re: best way to handle this in Python

Show key headers only | View raw

On Fri, Jul 20, 2012 at 4:34 AM, Rita <rmorgan466@gmail.com> wrote:
> Thats an interesting data structure Dennis. I will actually be running this
> type of query many times preferable in an ad-hoc environment. That makes it
> tough for sqlite3 since there will be several hundred thousand tuples.

Several hundred thousand is not an enormous number.  I think you're
underestimating sqlite3.  I just tried a test with one million tuples,
six colors per tuple (six million rows altogether).  Each row contains
a primary key, a timestamp, a color, and a count, with an index on the
timestamp column.  Building the database from scratch took about a
minute; adding the index took about another minute.  Incremental
updates would of course be much faster.  Queries like "select * from
data where timestamp between 500000 and 600000" return instantly (from
a user perspective).

Cheers,
Ian

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: best way to handle this in Python Ian Kelly <ian.g.kelly@gmail.com> - 2012-07-20 12:14 -0600
  Re: best way to handle this in Python Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-07-21 00:43 +0000

csiph-web