Groups > alt.folklore.computers > #235613

Re: Usenet-Rewind: search engine for Usenet messages back to 1981

Path	csiph.com!news.nrbbs.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx09.iad.POSTED!not-for-mail
From	Daniel <me@fuhgitabutit.com>
Newsgroups	alt.folklore.computers
Subject	Re: Usenet-Rewind: search engine for Usenet messages back to 1981
References	<XnsB4BF166DC48A6cstadler18hotmailcom@81.171.92.222>
Message-ID	<87bja158lb.fsf@rpi3> (permalink)
User-Agent	Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
Cancel-Lock	sha1:Cx0sI5VE5jIuxv3hHTy+uU/JP8w=
MIME-Version	1.0
Content-Type	text/plain
Lines	39
X-Complaints-To	abuse(at)newshosting.com
NNTP-Posting-Date	Sun, 13 Sep 2026 18:06:58 UTC
Organization	Newshosting.com - Highest quality at a great price! www.newshosting.com
Date	Sun, 13 Sep 2026 11:06:56 -0700
X-Received-Bytes	2345
Xref	csiph.com alt.folklore.computers:235613

Show key headers only | View raw

Craig Stadler <cstadler18@hotmail.com> writes:

> I spent the last year gathering and normalizing as many resources for
> Usenet messages and  built a search engine over public Usenet
> discussions going back to 1981. Posting here because this seemed like
> the one group that would actually care.
>
> https://www.usenet-rewind.com/
>
> Since Google Groups stopped indexing new Usenet content a
> while back, and search over its historical archive has always been
> rough- I wanted something that treats this as a real research
> archive -- full-text search, date-range/filtering,etc and most
> importantly as much Usenet data as possible going back as far as I could
> find. 
>
> The index currently holds (to date) roughly 980 million messages and is
> still growing. Sources are archival backups, data donations, and ongoing
> crawls of several thousand still-active public NNTP servers. Binary and
> yEnc-encoded content is stripped where possible to keep the index
> focused on text. 
>
> The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 
> 22.04.5 LTS & custom Python scripts, all on nvme disk.
>
> This is public archival content, same legal basis as DejaNews and
> Google Groups before it. There's a removal process for anyone who
> wants their own posts taken down, and author contact info is masked
> by default.

Is there any way you can create a gopher equivalent to your site?

Any chance that you'll adopt a public API? I would consider writing a
feature to my finger service + a door on my bbs to interface with it.

--
Daniel
sysop  | air & wave bbs
finger | info@bbs.airandwave.net

Back to alt.folklore.computers | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Usenet-Rewind: search engine for Usenet messages back to 1981 Craig Stadler <cstadler18@hotmail.com> - 2026-09-06 06:12 +0000
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 ram@zedat.fu-berlin.de (Stefan Ram) - 2026-09-06 07:18 +0000
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Nuno Silva <nunojsilva@invalid.invalid> - 2026-09-06 10:01 +0100
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Craig Stadler <cstadler18@hotmail.com> - 2026-09-06 16:55 +0000
      Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Nuno Silva <nunojsilva@invalid.invalid> - 2026-09-07 10:51 +0100
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Andy Burns <usenet@andyburns.uk> - 2026-09-06 10:31 +0100
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Peter Flass <Peter@Iron-Spring.com> - 2026-09-06 08:02 -0700
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Peter Flass <Peter@Iron-Spring.com> - 2026-09-06 08:12 -0700
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Mark Shaw <mshaw@panix.com> - 2026-09-10 00:52 +0000
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Daniel <me@fuhgitabutit.com> - 2026-09-13 11:06 -0700

csiph-web