Groups > comp.os.linux.advocacy > #349403

Re: Algorithm to find data range

From	owl <owl@rooftop.invalid>
Newsgroups	comp.os.linux.advocacy
Subject	Re: Algorithm to find data range
Date	2016-04-11 21:46 +0000
Organization	O.W.L.
Message-ID	<hgjei0a3.fauo@rooftop.invalid> (permalink)
References	<negfcb$4ee$1@dont-email.me> <sandman-2ada1150bc27e3d11b868223a447a75e@individual.net> <hgjdke902.ata34p@rooftop.invalid> <sandman-b3b70fc4cbc0697fbcd2cb88cf7e38a7@individual.net>

Show all headers | View raw

Sandman <mr@sandman.net> wrote:
> In article <hgjdke902.ata34p@rooftop.invalid>, owl wrote:
> 
>> > Sandman:
>> > Also, given restriction 3 above, there is no way for you to know
>> > what ID is at the middle of a sample. If you have 500k rows of
>> > data, you'd think that ID 250,000 would be in the middle, but
>> > since it was stipulated that ID could contain caps and not be
>> > continuous, you may have cut two thirds up in the series. In fact,
>> > ID 250,000 may be the very last ID according to rule #3. Or the
>> > first, for that matter.
>> 
>> I took it to mean 500,000 items, not necessarily ID 1 - ID 500,000.
> 
> Exactly, meaning that the ID could be anything, and since we were supposed to 
> fetch posts only by ID, it's impossible to fetch anything, right? For all we 
> know, the ID's could be 1,982,827 to 3,462,987 with enough gaps in the ID's to 
> amount ot 500,000 posts.

But the assumption is that he knows the first and last ID number and
the total count.  Also, assuming this is usenet posts, if there's a
gap at all, it's not going to be large.   


> 
> The only known variables are the number of posts, and that the posts have dates 
> ranging from 2004-05-03 to 2014-05-03, right?
> 
> So, basically, the data could be:
> 
> 2,765,982   2004-03-05
> 2,765,983   2004-03-05
> 2,765,984   2004-03-05
> 2,765,987   2004-03-05
> ..
> 4,756,234   2014-03-05
> 
> At this point it would be impossible to efficiently figure out what ID numbers 
> correlate to what dates if we can't query what the first and last ID is.
> 
> I currently have 172,946 posts in my usenet table, and if I were interested in 
> posts made between two dates, I would go:
> 
> : select * from usenet where date >= '2014-08-01' and date < '2014-09-01';
> 

He said he's not able to query by date.  He has to use the article ID
and test for date. 


> And I could get two posts back or several thousands, depending on traffic. And 
> since the date field is indexed, the query takes 15ms (and returned 3,513 
> posts).
> 

I believe he's querying a news server, not an SQL database.

Thread

Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 11:23 -0400
  Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-11 17:09 +0000
    Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 19:25 +0000
      Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-11 20:58 +0000
        Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 17:38 -0400
          Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 18:11 -0400
            Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 22:25 +0000
          Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 06:16 +0000
        Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 21:46 +0000
          Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 07:18 +0000
            Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 08:15 +0000
              Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 10:34 +0000
            Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:32 -0400
              Re: Algorithm to find data range Steve Carroll <fretwizzer@gmail.com> - 2016-04-12 10:39 -0700
                Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:58 -0400
                Re: Algorithm to find data range Steve Carroll <fretwizzer@gmail.com> - 2016-04-12 11:59 -0700
    Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 23:23 -0400
      Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 07:14 +0000
      Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 10:44 +0000
        Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 15:17 +0000
          Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-13 17:15 -0400
            Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-13 22:23 +0000
              Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-13 18:32 -0400
                Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-14 04:33 +0000
                Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-14 05:51 +0000
                Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-14 20:15 -0400
                Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-15 01:18 +0000
        Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:25 -0400
          Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 19:08 +0000
    Re: Algorithm to find data range vallor <vallor@cultnix.org> - 2016-04-12 04:40 +0000
  Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 18:06 +0000
  Re: Algorithm to find data range 7 <7@enemygadgets.com> - 2016-04-11 22:46 +0000
    Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-11 18:52 -0400
      Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:28 -0400
        Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-12 13:46 -0400
    Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 22:13 -0400
  Re: Algorithm to find data range Fabian Russell <fb@zen.info> - 2016-04-11 23:22 +0000
    Re: Algorithm to find data range vallor <vallor@cultnix.org> - 2016-04-12 00:06 +0000
      Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-11 20:26 -0400
      Re: Algorithm to find data range Chris Ahlstrom <OFeem1987@teleworm.us> - 2016-04-12 05:30 -0400
      Re: Algorithm to find data range chrisv <chrisv@nospam.invalid> - 2016-04-12 06:50 -0500
    Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 22:34 -0400
      Re: Algorithm to find data range Fabian Russell <fb@zen.info> - 2016-04-12 09:25 +0000
        Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:32 -0400
  Is it your own personal NNTP/Usenet server ? Jeff-Relf.Me <@.> - 2016-04-11 16:32 -0700
    Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 12:47 -0400
      Re: Algorithm to find data range Peter Köhlmann <peter-koehlmann@t-online.de> - 2016-04-12 20:46 +0200
        Re: Algorithm to find data range chrisv <chrisv@nospam.invalid> - 2016-04-12 13:59 -0500
        Re: Algorithm to find data range Silver Slimer <linux@sucks.balls> - 2016-04-12 17:08 -0400
      My "newsReader" (X.ZIP) is also a console. Jeff-Relf.Me <@.> - 2016-04-12 12:27 -0700
      My "newsReader" (X.ZIP) is also a console. Jeff-Relf.Me <@.> - 2016-04-12 12:31 -0700

csiph-web