Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > alt.folklore.computers > #235536 > unrolled thread

Usenet-Rewind: search engine for Usenet messages back to 1981

Back to article view | Back to alt.folklore.computers

Contents

  Usenet-Rewind: search engine for Usenet messages back to 1981 Craig Stadler <cstadler18@hotmail.com> - 2026-09-06 06:12 +0000
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 ram@zedat.fu-berlin.de (Stefan Ram) - 2026-09-06 07:18 +0000
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Nuno Silva <nunojsilva@invalid.invalid> - 2026-09-06 10:01 +0100
      Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Craig Stadler <cstadler18@hotmail.com> - 2026-09-06 16:55 +0000
        Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Nuno Silva <nunojsilva@invalid.invalid> - 2026-09-07 10:51 +0100
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Andy Burns <usenet@andyburns.uk> - 2026-09-06 10:31 +0100
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Peter Flass <Peter@Iron-Spring.com> - 2026-09-06 08:02 -0700
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Peter Flass <Peter@Iron-Spring.com> - 2026-09-06 08:12 -0700
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Mark Shaw <mshaw@panix.com> - 2026-09-10 00:52 +0000
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Daniel <me@fuhgitabutit.com> - 2026-09-13 11:06 -0700

#235536 — Usenet-Rewind: search engine for Usenet messages back to 1981

I spent the last year gathering and normalizing as many resources for
Usenet messages and  built a search engine over public Usenet
discussions going back to 1981. Posting here because this seemed like
the one group that would actually care.

https://www.usenet-rewind.com/

Since Google Groups stopped indexing new Usenet content a
while back, and search over its historical archive has always been
rough- I wanted something that treats this as a real research
archive -- full-text search, date-range/filtering,etc and most
importantly as much Usenet data as possible going back as far as I could
find. 

The index currently holds (to date) roughly 980 million messages and is
still growing. Sources are archival backups, data donations, and ongoing
crawls of several thousand still-active public NNTP servers. Binary and
yEnc-encoded content is stripped where possible to keep the index
focused on text. 

The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 
22.04.5 LTS & custom Python scripts, all on nvme disk.

This is public archival content, same legal basis as DejaNews and
Google Groups before it. There's a removal process for anyone who
wants their own posts taken down, and author contact info is masked
by default.

[toc] | [next] | [standalone]

#235537

Craig Stadler <cstadler18@hotmail.com> wrote or quoted:
>                         There's a removal process for anyone who
>wants their own posts taken down, and author contact info is masked
>by default.

  What about some automatic removals for posts exhibiting header
  lines such as:

X-No-Archive: Yes
Archive: no
X-No-Html: yes

  ?

[toc] | [prev] | [next] | [standalone]

#235538

On 2026-09-06, Craig Stadler wrote:

> I spent the last year gathering and normalizing as many resources for
> Usenet messages and  built a search engine over public Usenet
> discussions going back to 1981. Posting here because this seemed like
> the one group that would actually care.
>
> https://www.usenet-rewind.com/
>
> Since Google Groups stopped indexing new Usenet content a
> while back, and search over its historical archive has always been
> rough- I wanted something that treats this as a real research
> archive -- full-text search, date-range/filtering,etc and most
> importantly as much Usenet data as possible going back as far as I could
> find. 
>
> The index currently holds (to date) roughly 980 million messages and is
> still growing. Sources are archival backups, data donations, and ongoing
> crawls of several thousand still-active public NNTP servers. Binary and
> yEnc-encoded content is stripped where possible to keep the index
> focused on text. 
>
> The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 
> 22.04.5 LTS & custom Python scripts, all on nvme disk.
>
> This is public archival content, same legal basis as DejaNews and
> Google Groups before it. There's a removal process for anyone who
> wants their own posts taken down, and author contact info is masked
> by default.

Can you consider setting a width and/or height for SVG icons for better
compatibility? Or perhaps not use SVG icons at all?

I can get a paginated list of newsgroups, but they're not linkified, how
is this supposed to be used?

-- 
Nuno Silva

[toc] | [prev] | [next] | [standalone]

#235546

Nuno Silva <nunojsilva@invalid.invalid> wrote in news:117ja57$26mtg$1@dont-
email.me:

> Can you consider setting a width and/or height for SVG icons for better
> compatibility? Or perhaps not use SVG icons at all?
> 
> I can get a paginated list of newsgroups, but they're not linkified, how
> is this supposed to be used?
> 

I fixed the SVG size issue (I think!) and also newsgroups are now clickable 
but it shows ALL messages inthat group sorted by oldest first. Let me know 
if you had something else in mind, thanks for the feedback.

-Craig

[toc] | [prev] | [next] | [standalone]

#235548

On 2026-09-06, Craig Stadler wrote:

> Nuno Silva <nunojsilva@invalid.invalid> wrote in news:117ja57$26mtg$1@dont-
> email.me:
>
>> Can you consider setting a width and/or height for SVG icons for better
>> compatibility? Or perhaps not use SVG icons at all?
>> 
>> I can get a paginated list of newsgroups, but they're not linkified, how
>> is this supposed to be used?
>> 
>
> I fixed the SVG size issue (I think!) and also newsgroups are now clickable 
> but it shows ALL messages inthat group sorted by oldest first. Let me know 
> if you had something else in mind, thanks for the feedback.

Thanks, a (very) quick glance showed it is already much better than it
was on browsers which have the SVG size issue, and groups can be clicked
on!  :-)

I'll try to have a longer glance at it when time allows.

-- 
Nuno Silva

[toc] | [prev] | [next] | [standalone]

#235539

Craig Stadler wrote:

> this seemed like the one group that would actually care.
> https://www.usenet-rewind.com/
Thanks for your efforts on this, I've only had a quick poke at it which 
gave me results, I'll try it later ...

[toc] | [prev] | [next] | [standalone]

#235542

Thank you! This is definitely a worthwhile service. I'll have to try it out.

On 9/5/26 23:12, Craig Stadler wrote:
> I spent the last year gathering and normalizing as many resources for
> Usenet messages and  built a search engine over public Usenet
> discussions going back to 1981. Posting here because this seemed like
> the one group that would actually care.
> 
> https://www.usenet-rewind.com/
> 
> Since Google Groups stopped indexing new Usenet content a
> while back, and search over its historical archive has always been
> rough- I wanted something that treats this as a real research
> archive -- full-text search, date-range/filtering,etc and most
> importantly as much Usenet data as possible going back as far as I could
> find.
> 
> The index currently holds (to date) roughly 980 million messages and is
> still growing. Sources are archival backups, data donations, and ongoing
> crawls of several thousand still-active public NNTP servers. Binary and
> yEnc-encoded content is stripped where possible to keep the index
> focused on text.
> 
> The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu
> 22.04.5 LTS & custom Python scripts, all on nvme disk.
> 
> This is public archival content, same legal basis as DejaNews and
> Google Groups before it. There's a removal process for anyone who
> wants their own posts taken down, and author contact info is masked
> by default.

[toc] | [prev] | [next] | [standalone]

#235543

On 9/5/26 23:12, Craig Stadler wrote:
> I spent the last year gathering and normalizing as many resources for
> Usenet messages and  built a search engine over public Usenet
> discussions going back to 1981. Posting here because this seemed like
> the one group that would actually care.
> 
> https://www.usenet-rewind.com/
> 
> Since Google Groups stopped indexing new Usenet content a
> while back, and search over its historical archive has always been
> rough- I wanted something that treats this as a real research
> archive -- full-text search, date-range/filtering,etc and most
> importantly as much Usenet data as possible going back as far as I could
> find.
> 
> The index currently holds (to date) roughly 980 million messages and is
> still growing. Sources are archival backups, data donations, and ongoing
> crawls of several thousand still-active public NNTP servers. Binary and
> yEnc-encoded content is stripped where possible to keep the index
> focused on text.
> 
> The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu
> 22.04.5 LTS & custom Python scripts, all on nvme disk.
> 
> This is public archival content, same legal basis as DejaNews and
> Google Groups before it. There's a removal process for anyone who
> wants their own posts taken down, and author contact info is masked
> by default.

Too bad there isn't some way to get rid of the porn. Search for "cp-v" 
(with the quotes) retrieves the following as the first hit:
   VEB ** Submissive Daddy's Girl Whitney Does It All ** CP@V+
or else we need to know the rules for searches.

[toc] | [prev] | [next] | [standalone]

#235556

Craig Stadler <cstadler18@hotmail.com> wrote:

> I spent the last year gathering and normalizing as many resources for
> Usenet messages and  built a search engine over public Usenet
> discussions going back to 1981. Posting here because this seemed like
> the one group that would actually care.

> https://www.usenet-rewind.com/

Thank you for this. I had a look at my old stomping grounds at
rec.pets.dogs.behavior, and the first thing that came up in my
search was a response to someone who got taken in by the resident
newsgroup whackjob.

Good times, good times.

-- 
Mark Shaw                                        moc TOD liamg TA wahsnm
========================================================================
"Ironic that the two O's in 'cooperate' insisted on
having their own separate sounds...."                  -Nathan W. Pyle

[toc] | [prev] | [next] | [standalone]

#235613

Craig Stadler <cstadler18@hotmail.com> writes:

> I spent the last year gathering and normalizing as many resources for
> Usenet messages and  built a search engine over public Usenet
> discussions going back to 1981. Posting here because this seemed like
> the one group that would actually care.
>
> https://www.usenet-rewind.com/
>
> Since Google Groups stopped indexing new Usenet content a
> while back, and search over its historical archive has always been
> rough- I wanted something that treats this as a real research
> archive -- full-text search, date-range/filtering,etc and most
> importantly as much Usenet data as possible going back as far as I could
> find. 
>
> The index currently holds (to date) roughly 980 million messages and is
> still growing. Sources are archival backups, data donations, and ongoing
> crawls of several thousand still-active public NNTP servers. Binary and
> yEnc-encoded content is stripped where possible to keep the index
> focused on text. 
>
> The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 
> 22.04.5 LTS & custom Python scripts, all on nvme disk.
>
> This is public archival content, same legal basis as DejaNews and
> Google Groups before it. There's a removal process for anyone who
> wants their own posts taken down, and author contact info is masked
> by default.

Is there any way you can create a gopher equivalent to your site?

Any chance that you'll adopt a public API? I would consider writing a
feature to my finger service + a door on my bbs to interface with it.

--
Daniel
sysop  | air & wave bbs
finger | info@bbs.airandwave.net

[toc] | [prev] | [standalone]

Back to top | Article view | alt.folklore.computers

csiph-web