Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.advocacy > #349969
| From | owl <owl@rooftop.invalid> |
|---|---|
| Newsgroups | comp.os.linux.advocacy |
| Subject | Re: Algorithm to find data range |
| Date | 2016-04-15 01:18 +0000 |
| Organization | O.W.L. |
| Message-ID | <nbvmcjuf83.jilgda@rooftop.invalid> (permalink) |
| References | (6 earlier) <hjgi403.uut83gko@rooftop.invalid> <nemh84$vf4$1@dont-email.me> <nmbjd83.klguey@rooftop.invalid> <hjgo30a.rt4aef@rooftop.invalid> <nepblm$66a$1@dont-email.me> |
DFS <nospam@dfs.com> wrote:
> On 4/14/2016 1:51 AM, owl wrote:
>
>> I don't do python at all,
>
> But you did. How did you know to write that code?
>
>
>
>> but the following worked for passing the xpat.
>> I had already run once with just group command to get the range.
>> Of course, you'll need to supply the correct credentials and server.
>
>> import socket
>>
>> s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
>> s.connect(("localhost",119))
>> print s.recv(8192)
>> s.send("authinfo user USERNAME\n")
>> print s.recv(8192)
>> s.send("authinfo pass PASSWORD\n")
>> print s.recv(8192)
>> s.send("group comp.os.linux.advocacy\n")
>> print s.recv(8192)
>> s.send("xpat date 2152569-2234805 * 13 Apr 2016*\n")
>> print s.recv(8192)
>> print s.recv(8192)
>> s.close()
>
>
> Nice! Gave me 4 responses:
>
> * 200 mx02.eternal-september.org InterNetNews NNRP server INN 2.7.0
> (20151029 snapshot) ready (posting ok)
> * 381 Enter password
> * 281 Authentication succeeded
> * 211 368279 1 578403 comp.os.linux.advocacy
>
>
> Then I submitted
>
> s.send("xpat date 578000-578400 *Apr 2016*\n")
> print s.recv(8192)
> print s.recv(8192)
>
>
> and it gave dozens of hits, but the last one was cut off
>
> 578197 Tue, 12 Apr 2016 20:43:51 +0200
> 578198 Tue, 12 Apr 201
>
> I increased the recv buffer to 16384 but same thing.
>
> Some setting I need to make?
>
Probably really need to call recv() in a loop and buffer the data.
Take a look at this and other similar solutions:
http://code.activestate.com/recipes/408859-socketrecv-three-ways-to-turn-it-into-recvall/
The way I was doing it with bash/expect sometimes required that I set
the timeout higher to get the data. Fortunately, I would get null return
if expect did not get a match before the timeout, so it was obvious that
I needed to increase.
As basic and predicatable as this data is, you may can get by with
just multiple recv calls like below. Take note of the wildpat [^1-3]
for capturing day values less than 10. That is necessary or else it
won't get them all (i.e., "04" will not get "4" and "4" will not get
"04" values). In real code, of course, you'll need to calculate date
differences, grab the earliest ID from the initial date file, the latest
date from the offset date file, etc.
---------------------------
#!/usr/bin/python
import os
import socket
from tempfile import mkstemp
fd1, temp_path1 = mkstemp()
fd2, temp_path2 = mkstemp()
print "orig date IDs in here " + temp_path1
print "offset date IDs in here " + temp_path2
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.connect(("SERVER",119))
print s.recv(8192)
s.send("authinfo user USERNAME\n")
print s.recv(8192)
s.send("authinfo pass PASSWORD\n")
print s.recv(8192)
s.send("group comp.os.linux.advocacy\n")
print s.recv(8192)
s.send("xpat date 2152569-2234805 *[^1-3]2 Jul 2015*\n")
os.write(fd1,s.recv(32768))
os.write(fd1,s.recv(32768))
os.write(fd1,s.recv(32768))
os.close(fd1)
s.send("xpat date 2152569-2234805 *[^1-3]4 Mar 2016*\n")
os.write(fd2,s.recv(32768))
os.write(fd2,s.recv(32768))
os.write(fd2,s.recv(32768))
os.close(fd2)
s.close()
---------------------------
anon@lowtide:~/code/usenet$ time ./flah.py
orig date IDs in here /tmp/tmpjIe6Fq
offset date IDs in here /tmp/tmpC79Jjs
200 Check out http://www.altopia.com/ for info about NNTP access (posting ok).
381 PASS required
281 Ok
211 81986 2152993 2234978 comp.os.linux.advocacy
real 0m2.568s
user 0m0.044s
sys 0m0.016s
anon@lowtide:~/code/usenet$
anon@lowtide:~/code/usenet$ head /tmp/tmpjIe6Fq
221 date matches follow (NOV)
2187893 Thu, 02 Jul 2015 00:41:19 +0200
2187900 Thu, 02 Jul 2015 01:01:17 +0200
2187901 Thu, 02 Jul 2015 01:03:01 +0200
2187922 Thu, 2 Jul 2015 06:42:17 +0200
2187923 Thu, 2 Jul 2015 06:53:11 +0200
2187926 Thu, 2 Jul 2015 07:29:27 +0200
2187928 Thu, 2 Jul 2015 01:31:27 -0700 (PDT)
2187929 Thu, 2 Jul 2015 11:09:51 +0200
2187930 Thu, 2 Jul 2015 05:52:49 -0400
anon@lowtide:~/code/usenet$
anon@lowtide:~/code/usenet$ tail /tmp/tmpC79Jjs
2230279 Fri, 04 Mar 2016 17:35:51 -0800 (Seattle)
2230282 Fri, 4 Mar 2016 20:52:04 -0500
2230284 Fri, 4 Mar 2016 19:50:29 -0600
2230285 Fri, 4 Mar 2016 20:59:46 -0500
2230288 Fri, 4 Mar 2016 18:18:51 -0800 (PST)
2230292 Fri, 4 Mar 2016 21:44:11 -0500
2230293 Fri, 4 Mar 2016 21:46:07 -0500
2230294 Fri, 4 Mar 2016 21:50:04 -0500
2230296 Fri, 4 Mar 2016 21:10:53 -0600
2230299 Fri, 04 Mar 2016 19:41:13 -0800 (Seattle)
anon@lowtide:~/code/usenet$
Here you would want to parse out the 2187893 from /tmp/tmpjIe6Fq and
2230299 from /tmp/tmpC79Jjs.
Back to comp.os.linux.advocacy | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 11:23 -0400
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-11 17:09 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 19:25 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-11 20:58 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 17:38 -0400
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 18:11 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 22:25 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 06:16 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 21:46 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 07:18 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 08:15 +0000
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 10:34 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:32 -0400
Re: Algorithm to find data range Steve Carroll <fretwizzer@gmail.com> - 2016-04-12 10:39 -0700
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:58 -0400
Re: Algorithm to find data range Steve Carroll <fretwizzer@gmail.com> - 2016-04-12 11:59 -0700
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 23:23 -0400
Re: Algorithm to find data range Sandman <mr@sandman.net> - 2016-04-12 07:14 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 10:44 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 15:17 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-13 17:15 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-13 22:23 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-13 18:32 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-14 04:33 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-14 05:51 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-14 20:15 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-15 01:18 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:25 -0400
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-12 19:08 +0000
Re: Algorithm to find data range vallor <vallor@cultnix.org> - 2016-04-12 04:40 +0000
Re: Algorithm to find data range owl <owl@rooftop.invalid> - 2016-04-11 18:06 +0000
Re: Algorithm to find data range 7 <7@enemygadgets.com> - 2016-04-11 22:46 +0000
Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-11 18:52 -0400
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:28 -0400
Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-12 13:46 -0400
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 22:13 -0400
Re: Algorithm to find data range Fabian Russell <fb@zen.info> - 2016-04-11 23:22 +0000
Re: Algorithm to find data range vallor <vallor@cultnix.org> - 2016-04-12 00:06 +0000
Re: Algorithm to find data range Omar <omarsayeed@linuxmail.org> - 2016-04-11 20:26 -0400
Re: Algorithm to find data range Chris Ahlstrom <OFeem1987@teleworm.us> - 2016-04-12 05:30 -0400
Re: Algorithm to find data range chrisv <chrisv@nospam.invalid> - 2016-04-12 06:50 -0500
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-11 22:34 -0400
Re: Algorithm to find data range Fabian Russell <fb@zen.info> - 2016-04-12 09:25 +0000
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 13:32 -0400
Is it your own personal NNTP/Usenet server ? Jeff-Relf.Me <@.> - 2016-04-11 16:32 -0700
Re: Algorithm to find data range DFS <nospam@dfs.com> - 2016-04-12 12:47 -0400
Re: Algorithm to find data range Peter Köhlmann <peter-koehlmann@t-online.de> - 2016-04-12 20:46 +0200
Re: Algorithm to find data range chrisv <chrisv@nospam.invalid> - 2016-04-12 13:59 -0500
Re: Algorithm to find data range Silver Slimer <linux@sucks.balls> - 2016-04-12 17:08 -0400
My "newsReader" (X.ZIP) is also a console. Jeff-Relf.Me <@.> - 2016-04-12 12:27 -0700
My "newsReader" (X.ZIP) is also a console. Jeff-Relf.Me <@.> - 2016-04-12 12:31 -0700
Re: My "newsReader" (X.ZIP) is also a console. Unknown <dog@gmail.com> - 2016-06-14 00:48 +0000
csiph-web