Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > alt.folklore.computers > #235613

Re: Usenet-Rewind: search engine for Usenet messages back to 1981

Show all headers | View raw

Craig Stadler <cstadler18@hotmail.com> writes:

> I spent the last year gathering and normalizing as many resources for
> Usenet messages and  built a search engine over public Usenet
> discussions going back to 1981. Posting here because this seemed like
> the one group that would actually care.
>
> https://www.usenet-rewind.com/
>
> Since Google Groups stopped indexing new Usenet content a
> while back, and search over its historical archive has always been
> rough- I wanted something that treats this as a real research
> archive -- full-text search, date-range/filtering,etc and most
> importantly as much Usenet data as possible going back as far as I could
> find. 
>
> The index currently holds (to date) roughly 980 million messages and is
> still growing. Sources are archival backups, data donations, and ongoing
> crawls of several thousand still-active public NNTP servers. Binary and
> yEnc-encoded content is stripped where possible to keep the index
> focused on text. 
>
> The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 
> 22.04.5 LTS & custom Python scripts, all on nvme disk.
>
> This is public archival content, same legal basis as DejaNews and
> Google Groups before it. There's a removal process for anyone who
> wants their own posts taken down, and author contact info is masked
> by default.

Is there any way you can create a gopher equivalent to your site?

Any chance that you'll adopt a public API? I would consider writing a
feature to my finger service + a door on my bbs to interface with it.

--
Daniel
sysop  | air & wave bbs
finger | info@bbs.airandwave.net

Back to alt.folklore.computers | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Usenet-Rewind: search engine for Usenet messages back to 1981 Craig Stadler <cstadler18@hotmail.com> - 2026-09-06 06:12 +0000
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 ram@zedat.fu-berlin.de (Stefan Ram) - 2026-09-06 07:18 +0000
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Nuno Silva <nunojsilva@invalid.invalid> - 2026-09-06 10:01 +0100
    Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Craig Stadler <cstadler18@hotmail.com> - 2026-09-06 16:55 +0000
      Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Nuno Silva <nunojsilva@invalid.invalid> - 2026-09-07 10:51 +0100
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Andy Burns <usenet@andyburns.uk> - 2026-09-06 10:31 +0100
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Peter Flass <Peter@Iron-Spring.com> - 2026-09-06 08:02 -0700
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Peter Flass <Peter@Iron-Spring.com> - 2026-09-06 08:12 -0700
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Mark Shaw <mshaw@panix.com> - 2026-09-10 00:52 +0000
  Re: Usenet-Rewind: search engine for Usenet messages back to 1981 Daniel <me@fuhgitabutit.com> - 2026-09-13 11:06 -0700

csiph-web