Path: csiph.com!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Ben Collver <bencollver@tilde.pink>
Newsgroups: comp.infosystems.gopher
Subject: Re: Gopherpedia is down
Date: Thu, 20 Mar 2025 15:45:01 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 85
Message-ID: <vrhd5t$3h86u$1@dont-email.me>
References: <slrnvtleq1.1llb.anthk@openbsd.home>
Injection-Date: Thu, 20 Mar 2025 16:45:02 +0100 (CET)
Injection-Info: dont-email.me; posting-host="5c37c690a1103377e0386160f2557438"; logging-data="3711198"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/dv42dsHASqTptHfoXbo0/aVBVjehVjRY="
User-Agent: slrn/1.0.3 (Linux)
Cancel-Lock: sha1:ICyj1yTSIVBW6JGYtgS46pEn9aE=
Xref: csiph.com comp.infosystems.gopher:771

On 2025-03-20, anthk <anthk@openbsd.home> wrote:
> Now that gopher://gopherpedia.com it's down, is there any alternative?
>
> In case it gets back, I wish they implement the articles 
> as a gopher directory with two items, that's it:
> a link to the article itself and a directory with the article images.
>
> The article would have the images marked as [see figure1.png] or whatever.

Someone pointed out that gopherpedia.com uses clownflare for its
authoritative name servers.

    $ dig -t NS gopherpedia.com
    ...
    ns1.digitalocean.com.   103001  IN      A       172.64.52.210
    ns2.digitalocean.com.   103001  IN      A       172.64.53.21
    ns3.digitalocean.com.   103001  IN      A       172.64.49.209
    
    $ whois 172.64.52.210
    ...
    NetRange:       172.64.0.0 - 172.71.255.255
    CIDR:           172.64.0.0/13
    NetName:        CLOUDFLARENET
    NetHandle:      NET-172-64-0-0-1
    Parent:         NET172 (NET-172-0-0-0-0)
    NetType:        Direct Allocation
    OriginAS:       AS13335
    Organization:   Cloudflare, Inc. (CLOUD14)

After gopherpedia.com went down for many days, i chose to write a small
shell script to format Wikipedia pages as plain text.  I serve those
from my own gopher hole.  This script depends on two external programs.

* suckless `fold` command
<https://dl.suckless.org/sbase/>

* webdump
<https://codemadness.org/webdump.html>

    $ cat >gopherpoodia-fetch.sh <<__EOF__
#!/bin/sh
# Version 1
#
# Use suckless fold command instead of GNU coreutils.
# https://dl.suckless.org/sbase/sbase-0.1.tar.gz
#
# `fold` from GNU coreutils corrupts UTF-8.
# <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44975>
# <https://unix.stackexchange.com/questions/454044/
#  unicode-safe-alternative-for-fold/486428#486428>

page="$1"
if [ -z "$page" ]
then
    echo "Usage: gopherpoodia-fetch.sh [Wikipedia page title or URL]"
    echo ""
    echo "Converts Wikipedia page to plaintext document."
    echo ""
    exit 1
fi

base="https://en.wikipedia.org/wiki/"
title=${page##$base}
title=${title##/}
enctitle=$(echo "$title" | sed -e 's|/|%2F|g')
link="$base$enctitle"

agent="User-Agent: Gopherpoodia (user@stinkin.host)"
api="https://en.wikipedia.org/api/rest_v1"
url="$api/page/html/$enctitle"

outdir="public_gopher/wiki/"
safetitle=$(echo "$enctitle" |\
    sed -e 's|%2F|-|g' -e 's|\.\.|_|g' -e 's|:|_|g')
outfile="$outdir/$safetitle"

mkdir -p "$outdir"
curl -H "User-Agent: $agent" "$url" |\
    webdump -l -r -w 70 -b "$base"  |\
    sed -e 's|/\./|/|g'             |\
    fold.suckless -s -w 72 >"$outfile"

echo "" >>"$outfile"
echo "From: <$link>" >>"$outfile"
__EOF__