Re: Go-http-client

Path	csiph.com!weretis.net!feeder6.news.weretis.net!border-2.nntp.ord.giganews.com!nntp.giganews.com!Xl.tags.giganews.com!local-1.nntp.ord.giganews.com!nntp.brightview.co.uk!news.brightview.co.uk.POSTED!not-for-mail
NNTP-Posting-Date	Fri, 23 Feb 2024 11:58:34 +0000
From	Martin <News03@avisoft.f9.co.uk>
Subject	Re: Go-http-client
Newsgroups	comp.sys.acorn.networking
Date	Fri, 23 Feb 2024 11:56:13 +0000 (GMT)
Message-ID	<5b372e7c90News03@avisoft.f9.co.uk> (permalink)
References	<5b2b4df63fNews03@avisoft.f9.co.uk> <5b369f6248News03@avisoft.f9.co.uk> <5b37253c5anews*@Torrens.org>
User-Agent	Pluto/3.21a (RISC OS/5.29) NewsHound/v1.54
Organization	None
Lines	41
X-Usenet-Provider	http://www.giganews.com
X-Trace	sv3-VHvrXDc4WZreQi0ALmOg30N6USPEVLYGEHaFEdRffdiK2C8MVpm5RRJaLPFz3/NUYObb5oSX+fQcDjK!RbQUBHKiknqkKl1V9fgRdAV6Hvqd7tzmkWzgdOBYnDeoVqs99OoDjcbWK+xmc8dcnraDFgUerk1F!K9c=
X-Abuse-and-DMCA-Info	Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info	Otherwise we will be unable to process your complaint properly
X-Postfilter	1.3.40
Xref	csiph.com comp.sys.acorn.networking:5170

Show key headers only | View raw

In article <5b37253c5anews*@Torrens.org>,
   Richard Torrens (News) <News+19772@Torrens.org> wrote:
> In article <5b369f6248News03@avisoft.f9.co.uk>,
>    Martin <News03@avisoft.f9.co.uk> wrote:
> > The trickle continued, some days far outnumbering other requests.

> > But I have found a way to stop them! I added to my ./htaccess
> > file...

> > RewriteCond %{HTTP_USER_AGENT} "=Go-http-client/1.1"
> > RewriteRule .* - [F,L]

> > ... now returns 403 Forbidden. Stopped 260 in 12 hours yesterday.
> > This certainly works on PlusNet - may or may not on other ISPs.

> https://user-agents.net/string/go-http-client-1-1
> gives info on this. But these requests may not be evil. I would
> guess mostly neutral.

Of  1209 requests yesterday, 1117 were this user-agent -  over 96%.

They were from a wide variety of IP addresses, with anything from 2 to
30 requests each, to a similar subset of pages. 

None of them looked at robots/txt, so I would say that in my
experience, they were all spurious, probably malicious.

> There are so many bots and crawlers these days log files are of
> little practical use!

I have rarely looked at mine for ages ... until there was massive
increase in their daily sizes!

Anyway, they all get Forbidden from me now!

Martin

-- 
Martin Avison 
Note that unfortunately this email address will become invalid
without notice if (when) any spam is received.

Back to comp.sys.acorn.networking | Previous | Next — Previous in thread | Find similar

Thread

Go-http-client Martin <News03@avisoft.f9.co.uk> - 2024-01-31 10:25 +0000
  Re: Go-http-client Chris Hughes <news13@noonehere.co.uk> - 2024-01-31 11:47 +0000
    Re: Go-http-client Martin <News03@avisoft.f9.co.uk> - 2024-01-31 12:27 +0000
      Re: Go-http-client Chris Hughes <news13@noonehere.co.uk> - 2024-01-31 12:59 +0000
        Re: Go-http-client Martin <News03@avisoft.f9.co.uk> - 2024-01-31 13:11 +0000
  Re: Go-http-client Theo <theom+news@chiark.greenend.org.uk> - 2024-01-31 15:18 +0000
    Re: Go-http-client Martin <News03@avisoft.f9.co.uk> - 2024-01-31 16:56 +0000
      Re: Go-http-client Theo <theom+news@chiark.greenend.org.uk> - 2024-01-31 18:32 +0000
  Re: Go-http-client Martin <News03@avisoft.f9.co.uk> - 2024-02-22 09:53 +0000
    Re: Go-http-client "Richard Torrens (News)" <News+19772@Torrens.org> - 2024-02-23 10:15 +0000
      Re: Go-http-client Martin <News03@avisoft.f9.co.uk> - 2024-02-23 11:56 +0000

csiph-web