Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.advocacy > #349775
| From | owl <owl@rooftop.invalid> |
|---|---|
| Newsgroups | comp.os.linux.advocacy |
| Subject | Re: Algorithm to find data range |
| Date | 2016-04-14 04:33 +0000 |
| Organization | O.W.L. |
| Message-ID | <nmbjd83.klguey@rooftop.invalid> (permalink) |
| References | (3 earlier) <hjgi30ara.kgi4r3@rooftop.invalid> <fhjg0a9a.aer3@rooftop.invalid> <nemcmq$b8c$4@dont-email.me> <hjgi403.uut83gko@rooftop.invalid> <nemh84$vf4$1@dont-email.me> |
DFS <nospam@dfs.com> wrote:
> On 4/13/2016 6:23 PM, owl wrote:
>> DFS <nospam@dfs.com> wrote:
>>> On 4/12/2016 11:17 AM, owl wrote:
>>>
>>>
>>>> Performance is horrible, but it seems to work OK.
>>>> You'll think it's hung, but it's not.
>>>> Takes anywhere from 20-50 seconds to return.
>>>
>>> Sweet! I like a nice, relaxed program that finishes when it's damn good
>>> and ready.
>>>
>>>
>>>> Oh yeah, almost forgot: Linux FTW. ;)
>>>
>>> No doubt. Windows could never be that slow.
>>>
>>>
>>>
>>>> #!/bin/bash
>>> <snip>
>>>
>>>
>>> bash produces such beautiful code!
>>>
>>> You sent, you expected, you spawned, and you echoed... are you a dolphin?
>>>
>>
>> If fixed it. It's a lot faster now.
>> Also added option to set timeout in case the server is slow to return.
>> (Default timeout is set to 1 sec otherwise).
>>
>> anon@lowtide:~/code/usenet$ time ./blah.sh localhost cola 20150515 +180days
>> Earliest date searched: 15 May 2015
>> Latest date searched : 11 Nov 2015
>> First ID in range: 2178717 Fri, 15 May 2015 02:19:47 +0100 (BST)
>> Last ID in range : 2217475 Wed, 11 Nov 2015 21:33:03 -0600
>>
>> real 0m4.276s
>> user 0m0.028s
>> sys 0m0.032s
>> anon@lowtide:~/code/usenet$ time ./blah.sh localhost cola 20150515
>> Earliest date searched: 15 May 2015
>> Latest date searched : 15 May 2015
>> First ID in range: 2178717 Fri, 15 May 2015 02:19:47 +0100 (BST)
>> Last ID in range : 2178859 Fri, 15 May 2015 22:41:01 -0700
>>
>> real 0m2.775s
>> user 0m0.032s
>> sys 0m0.028s
>> anon@lowtide:~/code/usenet$
>
>
> Good job. 2.77 secs, that's something to shoot for.
>
> I can't (or can't figure out how to) use xpat with python, so I'll try
> different methods to get an ID range given a date range.
>
Does it not give you some way to write arbitrary string to the socket?
The server doesn't know what's connecting. It sends the same reponses to
whatever connects (telnet, netcat, your app, whatever). All you need is
a way to send commands and read its responses. If the python nntplib lets
you get the available range of IDs, then you have that range to build your
command string and send it with whatever write() function your language
provides.
Like the way I do it with telnet/expect:
send \"group ${GROUP}\r\"
expect \"211 \"
send \"${XPAT_EARLIEST}\r\"
The "${XPAT_EARLIEST}" is just a string like:
"xpat date 1111111-2222222 * [^0-3]2 May 2015"
(for day number < 10)
or
"xpat date 1111111-2222222 * 12 May 2015"
(for day number >= 10)
> Discovered as of last night on eternal-september:
> - article ID range is 1 to 578254
> - article count is 368130
> - huge gap: nothing from ID 1 (Nov 2007) to ID 201860 (Oct 2010)
>
5.5 years is still a nice batch available.
Back to comp.os.linux.advocacy | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 11:23 -0400
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-11 17:09 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 19:25 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-11 20:58 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 17:38 -0400
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 18:11 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 22:25 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 06:16 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 21:46 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 07:18 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 08:15 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 10:34 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:32 -0400
Re: Algorithm to find data range Steve Carroll <fretwizzer@gmail.com> - 2016-04-12 10:39 -0700
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:58 -0400
Re: Algorithm to find data range Steve Carroll <fretwizzer@gmail.com> - 2016-04-12 11:59 -0700
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 23:23 -0400
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 07:14 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 10:44 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 15:17 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-13 17:15 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-13 22:23 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-13 18:32 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-14 04:33 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-14 05:51 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-14 20:15 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-15 01:18 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:25 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 19:08 +0000
Re: Algorithm to find data range vallor <vallor@cultnix.org> - 2016-04-12 04:40 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 18:06 +0000
Re: Algorithm to find data range 7 <7@enemygadgets.com> - 2016-04-11 22:46 +0000
Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-11 18:52 -0400
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:28 -0400
Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-12 13:46 -0400
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 22:13 -0400
Re: Algorithm to find data range Fabian Russell <fb@zen.info> - 2016-04-11 23:22 +0000
Re: Algorithm to find data range vallor <vallor@cultnix.org> - 2016-04-12 00:06 +0000
Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-11 20:26 -0400
Re: Algorithm to find data range Chris Ahlstrom <OFeem1987@teleworm.us> - 2016-04-12 05:30 -0400
Re: Algorithm to find data range chrisv <chrisv@nospam.invalid> - 2016-04-12 06:50 -0500
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 22:34 -0400
Re: Algorithm to find data range Fabian Russell <fb@zen.info> - 2016-04-12 09:25 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:32 -0400
Is it your own personal NNTP/Usenet server ? Jeff-Relf.Me <@.> - 2016-04-11 16:32 -0700
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 12:47 -0400
Re: Algorithm to find data range Peter Köhlmann <peter-koehlmann@t-online.de> - 2016-04-12 20:46 +0200
Re: Algorithm to find data range chrisv <chrisv@nospam.invalid> - 2016-04-12 13:59 -0500
Re: Algorithm to find data range Silver Slimer <linux@sucks.balls> - 2016-04-12 17:08 -0400
My "newsReader" (X.ZIP) is also a console. Jeff-Relf.Me <@.> - 2016-04-12 12:27 -0700
My "newsReader" (X.ZIP) is also a console. Jeff-Relf.Me <@.> - 2016-04-12 12:31 -0700
Re: My "newsReader" (X.ZIP) is also a console. Unknown <dog@gmail.com> - 2016-06-14 00:48 +0000
csiph-web