Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #219457 > unrolled thread

new, not nice web bots disposal

Back to article view | Back to linux.debian.user

Contents

  new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 10:00 +0100
    Re: new, not nice web bots disposal john doe <johndoe65534@mail.com> - 2020-02-26 10:20 +0100
      Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 15:10 +0100
        Re: new, not nice web bots disposal Dan Ritter <dsr@randomstring.org> - 2020-02-26 15:30 +0100
    Re: new, not nice web bots disposal Jonas Smedegaard <jonas@jones.dk> - 2020-02-26 10:30 +0100
      Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 15:20 +0100
    Re: new, not nice web bots disposal Dan Ritter <dsr@randomstring.org> - 2020-02-26 12:20 +0100
      Re: new, not nice web bots disposal Roger Price <debian@rogerprice.org> - 2020-02-26 12:50 +0100
        Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 15:30 +0100
        Re: new, not nice web bots disposal Klaus Singvogel <deb-user-ml@singvogel.net> - 2020-02-26 15:30 +0100
    Re: new, not nice web bots disposal Greg Wooledge <wooledg@eeg.ccf.org> - 2020-02-26 14:30 +0100
      Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 15:50 +0100
        Re: new, not nice web bots disposal Roger Price <debian@rogerprice.org> - 2020-02-26 16:50 +0100
          Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 20:10 +0100
    Re: new, not nice web bots disposal Lee <ler762@gmail.com> - 2020-02-26 20:00 +0100
      Re: new, not nice web bots disposal Reco <recoverym4n@enotuniq.net> - 2020-02-26 20:00 +0100
        Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 20:20 +0100
          Re: new, not nice web bots disposal Reco <recoverym4n@enotuniq.net> - 2020-02-26 20:30 +0100
            Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 20:50 +0100
        Re: new, not nice web bots disposal <tomas@tuxteam.de> - 2020-02-26 22:10 +0100
          Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 22:50 +0100
            Re: new, not nice web bots disposal Lee <ler762@gmail.com> - 2020-02-27 05:30 +0100
              Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-27 06:10 +0100
                Re: new, not nice web bots disposal <tomas@tuxteam.de> - 2020-02-27 10:00 +0100
                  Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-27 14:30 +0100
                    Re: new, not nice web bots disposal <tomas@tuxteam.de> - 2020-02-27 15:20 +0100
                      Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-27 15:40 +0100
                        Re: new, not nice web bots disposal <tomas@tuxteam.de> - 2020-02-27 16:00 +0100
                          Re: new, not nice web bots disposal Dan Ritter <dsr@randomstring.org> - 2020-02-27 16:20 +0100
                        OT: LIbrarian (was: Re: new, not nice web bots disposal) rhkramer@gmail.com - 2020-02-27 16:40 +0100
              Re: new, not nice web bots disposal <tomas@tuxteam.de> - 2020-02-27 10:00 +0100
                Re: new, not nice web bots disposal Lee <ler762@gmail.com> - 2020-02-27 16:10 +0100
                  Re: new, not nice web bots disposal tomas@tuxteam.de - 2020-02-27 16:20 +0100
                    Re: new, not nice web bots disposal Lee <ler762@gmail.com> - 2020-02-27 16:40 +0100
                  Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-27 19:40 +0100
            Re: new, not nice web bots disposal <tomas@tuxteam.de> - 2020-02-27 10:00 +0100
              Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-27 14:20 +0100
                Re: new, not nice web bots disposal <tomas@tuxteam.de> - 2020-02-27 15:20 +0100
                Re: new, not nice web bots disposal songbird <songbird@anthive.com> - 2020-02-28 06:20 +0100
      Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 20:20 +0100
        Re: new, not nice web bots disposal Lee <ler762@gmail.com> - 2020-02-26 20:50 +0100
    Re: new, not nice web bots disposal deloptes <deloptes@gmail.com> - 2020-02-26 21:00 +0100
      Re: new, not nice web bots disposal Gene Heskett <gheskett@shentel.net> - 2020-02-26 22:30 +0100

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#219508

On Wednesday 26 February 2020 16:00:35 tomas@tuxteam.de wrote:

> On Wed, Feb 26, 2020 at 09:54:09PM +0300, Reco wrote:
> > 	Hi.
> >
> > On Wed, Feb 26, 2020 at 01:50:40PM -0500, Lee wrote:
>
> [...]
>
> > > Have you considered REJECT instead of DROP?
> >
> > A neat idea for your LAN. A bad idea in this case.
>
> Exactly.
>
> > You *want* that other side to retry, wasting their time instead of
> > spamming their target. In fact, one should consider using TARPIT
> > instead of a DROP here.

My copy of iptables-extensions makes zero mention of TARPIT.

> Moreover: you don't want the other side to even know that you're
> there. The less info you give away the better.

My reasoning too. I'd much druther be a black hole that doesn't even have 
any Hawking Radiation. But I've no info that such a beast exists 
anyplace in the universe. There is info in the fact of there not being 
any response.
>
> In a LAN, however, REJECT is far better: you want the other side
> to know that you're there, but not talking.

I'd call this a WAN since its intended to go out on the internet.
And I am the only user inside this LAN.

In that event, and given that a /24 rule caught them, how many out of 
that /24 get the reject message?

The iptables docs aren't that talkative... No need to bang on the other 
255 customers in that block that are not offenders.  That wouldn't be at 
all neighborly. And likely would make the situation much worse.

> Cheers
> -- t


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#219519

On 2/26/20, Gene Heskett <gheskett@shentel.net> wrote:
> On Wednesday 26 February 2020 16:00:35 tomas@tuxteam.de wrote:
>
>> On Wed, Feb 26, 2020 at 09:54:09PM +0300, Reco wrote:
>> > 	Hi.
>> >
>> > On Wed, Feb 26, 2020 at 01:50:40PM -0500, Lee wrote:
>>
>> [...]
>>
>> > > Have you considered REJECT instead of DROP?
>> >
>> > A neat idea for your LAN. A bad idea in this case.
>>
>> Exactly.
>>
>> > You *want* that other side to retry, wasting their time instead of
>> > spamming their target. In fact, one should consider using TARPIT
>> > instead of a DROP here.
>
> My copy of iptables-extensions makes zero mention of TARPIT.
>
>> Moreover: you don't want the other side to even know that you're
>> there. The less info you give away the better.
>
> My reasoning too.

You're advertising your web server in your sig.  The "other side"
ALREADY KNOWS you have a web server there.

If you're going to advertise your presence on the web it seems
pointless to pretend that you're not there.  And the bots you'd be
REJECTing are the ones that have ignored your robots.txt file, so why
not just tell them to go away instead of putting up with their
retries?

> I'd much druther be a black hole that doesn't even have
> any Hawking Radiation. But I've no info that such a beast exists
> anyplace in the universe. There is info in the fact of there not being
> any response.
>>
>> In a LAN, however, REJECT is far better: you want the other side
>> to know that you're there, but not talking.
>
> I'd call this a WAN since its intended to go out on the internet.
> And I am the only user inside this LAN.
>
> In that event, and given that a /24 rule caught them, how many out of
> that /24 get the reject message?

However many hit the REJECT rule.  The iptables rule is going to send
a RST to anything in that /24 tries to access your server.  The other
hosts in that /24 that aren't trying to access your server won't get
anything from you.

Regards,
Lee

[toc] | [prev] | [next] | [standalone]

#219521

On Wednesday 26 February 2020 23:25:53 Lee wrote:

> On 2/26/20, Gene Heskett <gheskett@shentel.net> wrote:
> > On Wednesday 26 February 2020 16:00:35 tomas@tuxteam.de wrote:
> >> On Wed, Feb 26, 2020 at 09:54:09PM +0300, Reco wrote:
> >> > 	Hi.
> >> >
> >> > On Wed, Feb 26, 2020 at 01:50:40PM -0500, Lee wrote:
> >>
> >> [...]
> >>
> >> > > Have you considered REJECT instead of DROP?
> >> >
> >> > A neat idea for your LAN. A bad idea in this case.
> >>
> >> Exactly.
> >>
> >> > You *want* that other side to retry, wasting their time instead
> >> > of spamming their target. In fact, one should consider using
> >> > TARPIT instead of a DROP here.
> >
> > My copy of iptables-extensions makes zero mention of TARPIT.
> >
> >> Moreover: you don't want the other side to even know that you're
> >> there. The less info you give away the better.
> >
> > My reasoning too.
>
> You're advertising your web server in your sig.  The "other side"
> ALREADY KNOWS you have a web server there.
>
This is true...

> If you're going to advertise your presence on the web it seems
> pointless to pretend that you're not there.  And the bots you'd be
> REJECTing are the ones that have ignored your robots.txt file, so why
> not just tell them to go away instead of putting up with their
> retries?

What if they ignore that RST too?

> > I'd much druther be a black hole that doesn't even have
> > any Hawking Radiation. But I've no info that such a beast exists
> > anyplace in the universe. There is info in the fact of there not
> > being any response.
> >
> >> In a LAN, however, REJECT is far better: you want the other side
> >> to know that you're there, but not talking.
> >
> > I'd call this a WAN since its intended to go out on the internet.
> > And I am the only user inside this LAN.
> >
> > In that event, and given that a /24 rule caught them, how many out
> > of that /24 get the reject message?
>
> However many hit the REJECT rule.  The iptables rule is going to send
> a RST to anything in that /24 tries to access your server.  The other
> hosts in that /24 that aren't trying to access your server won't get
> anything from you.

Good, you guys are beginning to make sense. Done.

> Regards,
> Lee


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#219530

[Multipart message — attachments visible in raw view] — view raw

On Wed, Feb 26, 2020 at 11:59:27PM -0500, Gene Heskett wrote:

[...]

> What if they ignore that RST too?

Read -- at least skim that wikipedia article (oh, I forgot the ref
in my other mail upthread, sorry. Here it is:

https://en.wikipedia.org/wiki/Transmission_Control_Protocol

Cheers
-- t

[toc] | [prev] | [next] | [standalone]

#219545

On Thursday 27 February 2020 03:56:07 tomas@tuxteam.de wrote:

> On Wed, Feb 26, 2020 at 11:59:27PM -0500, Gene Heskett wrote:
>
> [...]
>
> > What if they ignore that RST too?
>
> Read -- at least skim that wikipedia article (oh, I forgot the ref
> in my other mail upthread, sorry. Here it is:
>
> https://en.wikipedia.org/wiki/Transmission_Control_Protocol

People write software, dishonest people write software to do what they 
want to achieve.  And I've walked around in OP software enough in the 
past to have seen stuff I know damned well had to be on purpose.
>
> Cheers
> -- t


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#219551

[Multipart message — attachments visible in raw view] — view raw

On Thu, Feb 27, 2020 at 08:22:53AM -0500, Gene Heskett wrote:
> On Thursday 27 February 2020 03:56:07 tomas@tuxteam.de wrote:
> 
> > On Wed, Feb 26, 2020 at 11:59:27PM -0500, Gene Heskett wrote:
> >
> > [...]
> >
> > > What if they ignore that RST too?
> >
> > Read -- at least skim that wikipedia article (oh, I forgot the ref
> > in my other mail upthread, sorry. Here it is:
> >
> > https://en.wikipedia.org/wiki/Transmission_Control_Protocol
> 
> People write software, dishonest people write software to do what they 
> want to achieve.  And I've walked around in OP software enough in the 
> past to have seen stuff I know damned well had to be on purpose.

Had you followed my advice, you'd know by now that the lowest layer
of your network stack in Linux will throw away any packets arriving
from the other side if "they ignore the RST".

Cheers
-- t

[toc] | [prev] | [next] | [standalone]

#219555

On Thursday 27 February 2020 09:18:55 tomas@tuxteam.de wrote:

> On Thu, Feb 27, 2020 at 08:22:53AM -0500, Gene Heskett wrote:
> > On Thursday 27 February 2020 03:56:07 tomas@tuxteam.de wrote:
> > > On Wed, Feb 26, 2020 at 11:59:27PM -0500, Gene Heskett wrote:
> > >
> > > [...]
> > >
> > > > What if they ignore that RST too?
> > >
> > > Read -- at least skim that wikipedia article (oh, I forgot the ref
> > > in my other mail upthread, sorry. Here it is:
> > >
> > > https://en.wikipedia.org/wiki/Transmission_Control_Protocol
> >
> > People write software, dishonest people write software to do what
> > they want to achieve.  And I've walked around in OP software enough
> > in the past to have seen stuff I know damned well had to be on
> > purpose.
>
> Had you followed my advice, you'd know by now that the lowest layer
> of your network stack in Linux will throw away any packets arriving
> from the other side if "they ignore the RST".
>
This may well be true. But I still doubt its available AT THAT LIBRARY. 
Basically she insists on haveing a receipt that proves the library has 
legally purchased anything offered to lend. Snotty little thing, about 
50ish and maybe 4'10" if she was stretched on a rack for a week. She 
uses attitude to make up for short stature. I guess she figures its a 
lifetime job.

> Cheers


> -- t


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#219560

[Multipart message — attachments visible in raw view] — view raw

On Thu, Feb 27, 2020 at 09:35:44AM -0500, Gene Heskett wrote:
> On Thursday 27 February 2020 09:18:55 tomas@tuxteam.de wrote:

[...]

> > Had you followed my advice, you'd know by now that the lowest layer
> > of your network stack in Linux will throw away any packets arriving
> > from the other side if "they ignore the RST".
> >
> This may well be true. But I still doubt its available AT THAT LIBRARY. 

I was talking about the Wikipedia article. Recommended.

> Basically she insists on haveing a receipt that proves the library has 
> legally purchased anything offered to lend. Snotty little thing, about 
> 50ish and maybe 4'10" if she was stretched on a rack for a week. She 
> uses attitude to make up for short stature. I guess she figures its a 
> lifetime job.

There must be alternatives. For example Barnes & Noble has it here [1].
(No, I'm not linking to Amazon, but you know how to do that, I guess).

Volume 1 is the interesting bit. But the Wikipedia article will do.

Cheers

[1] https://www.barnesandnoble.com/w/tcp-ip-illustrated-volume-1-kevin-r-fall/1111573782
-- tomás

[toc] | [prev] | [next] | [standalone]

#219569

tomas@tuxteam.de wrote: 
> On Thu, Feb 27, 2020 at 09:35:44AM -0500, Gene Heskett wrote:
> > This may well be true. But I still doubt its available AT THAT LIBRARY. 
> 
> I was talking about the Wikipedia article. Recommended.
> There must be alternatives. For example Barnes & Noble has it here [1].
> (No, I'm not linking to Amazon, but you know how to do that, I guess).
> 
> Volume 1 is the interesting bit. But the Wikipedia article will do.
> 
> [1] https://www.barnesandnoble.com/w/tcp-ip-illustrated-volume-1-kevin-r-fall/1111573782

For what it's worth, this was one of my course texts back many
years ago, and I still keep it around.

Mostly I loan it to other people.

-dsr-

[toc] | [prev] | [next] | [standalone]

#219574 — OT: LIbrarian (was: Re: new, not nice web bots disposal)

On Thursday, February 27, 2020 09:35:44 AM Gene Heskett wrote:
> This may well be true. But I still doubt its available AT THAT LIBRARY.
> Basically she insists on haveing a receipt that proves the library has
> legally purchased anything offered to lend. 

I wonder why she does that?  I wonder if she had some bad experience with 
copyright law in the past, and is now very shy?

It might pay somebody in the library's geographic service area to talk to her 
gently about this and try to see if she will change her attitude (about 
receipts).

It might even affect her snot.

> Snotty little thing, about
> 50ish and maybe 4'10" if she was stretched on a rack for a week. She
> uses attitude to make up for short stature. I guess she figures its a
> lifetime job.

[toc] | [prev] | [next] | [standalone]

#219529

[Multipart message — attachments visible in raw view] — view raw

On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:

[...]

> You're advertising your web server in your sig.  The "other side"
> ALREADY KNOWS you have a web server there.

If that "other side" is reading your emails, that is.

Not a likely scenario if that "other side" is some malware
running in some whatever-of-things lightbulb or cat feeder.

Cheers
-- t

[toc] | [prev] | [next] | [standalone]

#219565

On 2/27/20, tomas@tuxteam.de  wrote:
> On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:
>
> [...]
>
>> You're advertising your web server in your sig.  The "other side"
>> ALREADY KNOWS you have a web server there.
>
> If that "other side" is reading your emails, that is.
>
> Not a likely scenario if that "other side" is some malware
> running in some whatever-of-things lightbulb or cat feeder.

This thread is NOT about likely scenarios; we're talking about

| over the last 90 days or so, we seem to have been plauged with a new
| breed of bots scanning our web pages, and they are not just indexing our
| web pages I don't mind that, but they are ignoring our robots.txt and
| are  mirroring anything apache2 can reach, including stuff thats there
| but not reachable by a normal browser just looking around and clicking
| on links.  Its annoying as hell and when you're out in the pucker-brush
| on a 10 megabit ADSL, eats up ones available upload bandwidth of about
| 275kbytes/s.  According to my cable billing, these A-H's used over 100Gb
| of my bandwidth in Nov 2019. That describes in printable language as a
| DDOS in my vocabulary.
|
| So I asked a few questions and wrote some little 2-3 line scripts after
| putting a tail on /var/lib/httpd/other_vhosts_access.log, which logs
| enough info you can generally identify the bots with it.
|
| I have since have generated 49 iptables rules that have blocked 99% of
| them.

**in this case** is it better to have DROP or REJECT on the iptable rules?

I'm saying it might be better to reject than drop.  Watch the logs and
if the A-H's ignore RSTs then go back to drop.

Regards,
Lee

[toc] | [prev] | [next] | [standalone]

#219566

[Multipart message — attachments visible in raw view] — view raw

On Thu, Feb 27, 2020 at 10:07:18AM -0500, Lee wrote:
> On 2/27/20, tomas@tuxteam.de  wrote:
> > On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:
> >
> > [...]
> >
> >> You're advertising your web server in your sig.  The "other side"
> >> ALREADY KNOWS you have a web server there.
> >
> > If that "other side" is reading your emails, that is.
> >
> > Not a likely scenario if that "other side" is some malware
> > running in some whatever-of-things lightbulb or cat feeder.
> 
> This thread is NOT about likely scenarios; we're talking about
> 
> | over the last 90 days or so, we seem to have been plauged with a new
> | breed of bots [...]

> I'm saying it might be better to reject than drop.  Watch the logs and
> if the A-H's ignore RSTs then go back to drop.

I disagree. DROP is the right thing on the Big Bad Internet. And I explained
upthread why: better to not let "them" know that you even are there.

Another reason: one less useless packet crossing the Internet.

Cheers
-- t

[toc] | [prev] | [next] | [standalone]

#219573

On 2/27/20, tomas@tuxteam.de <tomas@tuxteam.de> wrote:
> On Thu, Feb 27, 2020 at 10:07:18AM -0500, Lee wrote:
>> On 2/27/20, tomas@tuxteam.de  wrote:
>> > On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:
>> >
>> > [...]
>> >
>> >> You're advertising your web server in your sig.  The "other side"
>> >> ALREADY KNOWS you have a web server there.
>> >
>> > If that "other side" is reading your emails, that is.
>> >
>> > Not a likely scenario if that "other side" is some malware
>> > running in some whatever-of-things lightbulb or cat feeder.
>>
>> This thread is NOT about likely scenarios; we're talking about
>>
>> | over the last 90 days or so, we seem to have been plauged with a new
>> | breed of bots [...]
>
>> I'm saying it might be better to reject than drop.  Watch the logs and
>> if the A-H's ignore RSTs then go back to drop.
>
> I disagree. DROP is the right thing on the Big Bad Internet. And I
> explained
> upthread why: better to not let "them" know that you even are there.

They already know his server is there!!!  Gene created the iptable
rules **because** they were eating up too much of his bandwidth.

> Another reason: one less useless packet crossing the Internet.

vs. all the useless retries?

What I'm saying is that **it's possible** they'll respect a RST and
stop trying to connect to his server (in which case it's not a
'useless packet crossing the Internet.')  If they do respect RSTs then
Gene wins and if they don't (he needs to monitor his logs to see if
they keep retrying or not) then go back to DROP

Lee

[toc] | [prev] | [next] | [standalone]

#219588

On Thursday 27 February 2020 10:07:18 Lee wrote:

> On 2/27/20, tomas@tuxteam.de  wrote:
> > On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:
> >
> > [...]
> >
> >> You're advertising your web server in your sig.  The "other side"
> >> ALREADY KNOWS you have a web server there.
> >
> > If that "other side" is reading your emails, that is.
> >
> > Not a likely scenario if that "other side" is some malware
> > running in some whatever-of-things lightbulb or cat feeder.
>
> This thread is NOT about likely scenarios; we're talking about
>
> | over the last 90 days or so, we seem to have been plauged with a new
> | breed of bots scanning our web pages, and they are not just indexing
> | our web pages I don't mind that, but they are ignoring our
> | robots.txt and are  mirroring anything apache2 can reach, including
> | stuff thats there but not reachable by a normal browser just looking
> | around and clicking on links.  Its annoying as hell and when you're
> | out in the pucker-brush on a 10 megabit ADSL, eats up ones available
> | upload bandwidth of about 275kbytes/s.  According to my cable
> | billing, these A-H's used over 100Gb of my bandwidth in Nov 2019.
> | That describes in printable language as a DDOS in my vocabulary.
> |
> | So I asked a few questions and wrote some little 2-3 line scripts
> | after putting a tail on /var/lib/httpd/other_vhosts_access.log,
> | which logs enough info you can generally identify the bots with it.
> |
> | I have since have generated 49 iptables rules that have blocked 99%
> | of them.
>
> **in this case** is it better to have DROP or REJECT on the iptable
> rules?
>
> I'm saying it might be better to reject than drop.  Watch the logs and
> if the A-H's ignore RSTs then go back to drop.
>
> Regards,
> Lee

Okkaaayyyy. I rebooted about 14 hours back and restarted after switching 
it all to reject which says is now sending a "reject-with 
icmp-port-unreachable" msg.

If they are obeying the REJECT, one hit should do it, right?  In 14 hours 
of uptime, 4 have hit a given rule more than once, up to 9 times.  That 
looks like they are ignoreing the REJECT's to me.  The overall traffic 
is more frequent, but is not at a nuisance level, yet.

Thanks Lee

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#219531

[Multipart message — attachments visible in raw view] — view raw

On Wed, Feb 26, 2020 at 04:40:45PM -0500, Gene Heskett wrote:

[...]

> My reasoning too. I'd much druther be a black hole that doesn't even have 
> any Hawking Radiation.

The bigger the hole, the less Hawking radiation :)

[...]

> In that event, and given that a /24 rule caught them, how many out of 
> that /24 get the reject message?

Since the REJECT is part of the TCP connection establishment, only
one is the target of this response (how many actually /can/ see it
in their local segment is anyone's guess, but these days of intelligent
switches, segments tend to be small-ish).

> The iptables docs aren't that talkative... No need to bang on the other 
> 255 customers in that block that are not offenders.  That wouldn't be at 
> all neighborly. And likely would make the situation much worse.

Go, no: run to your Internet Library [1] and read up until the section
"Connection Establishment" (no worries: it's early in that book).

And for people behind restricted Internets: Wikipedia is downloadable!

Alternatively go to your paper library and present yourself with one
copy of R.W. Stevens "TCP-IP Illustrated". Not the most modern, but
one of the best.

Cheers
-- t

[toc] | [prev] | [next] | [standalone]

#219543

On Thursday 27 February 2020 03:50:34 tomas@tuxteam.de wrote:

> On Wed, Feb 26, 2020 at 04:40:45PM -0500, Gene Heskett wrote:
>
> [...]
>
> > My reasoning too. I'd much druther be a black hole that doesn't even
> > have any Hawking Radiation.
>
> The bigger the hole, the less Hawking radiation :)
>
> [...]
>
> > In that event, and given that a /24 rule caught them, how many out
> > of that /24 get the reject message?
>
> Since the REJECT is part of the TCP connection establishment, only
> one is the target of this response (how many actually /can/ see it
> in their local segment is anyone's guess, but these days of
> intelligent switches, segments tend to be small-ish).
>
> > The iptables docs aren't that talkative... No need to bang on the
> > other 255 customers in that block that are not offenders.  That
> > wouldn't be at all neighborly. And likely would make the situation
> > much worse.
>
> Go, no: run to your Internet Library [1] and read up until the section
> "Connection Establishment" (no worries: it's early in that book).
>
> And for people behind restricted Internets: Wikipedia is downloadable!
>
> Alternatively go to your paper library and present yourself with one
> copy of R.W. Stevens "TCP-IP Illustrated". Not the most modern, but
> one of the best.

This library is a joke, the librarian is scared shitless of copyright 
law.  When I retired, I had an 18 year collection of McGraw-Hill's 
Electronics magazine, from which anybody that could read, could get 
himself the equ of the best education in electronics available anyplace 
on the planet, hopeing I might plant a seed in some youngster. Several 
hundred lbs of it since a year was at least 5-6" on the shelf. I went 
back to see how they were displayed about a week later, they weren't, 
the bitch had filed them in the trash bin and they'd already been picked 
up.  I went home and cry'd. To me, that was the equ of burning the 
Library of Alexandria.  That magazine was at least half my education in 
electronics. At that point, in 18 years as CE at WDTV, one piece of gear 
had been returned to the maker for repair, a Canuk outfit that didn't an 
american to fix his stuff, I kept everything else ticking along and the 
case cow fresh.  And you'd be amazed at how quick a news dept. can 
demolish a $30,000 camera.

Sure, I'm an old fart now, but as a child I had "The Nack" and I quit 
school over a health issue in the 9th grade, and went to work fixing the 
then (1948) new-fangled things called televisions, but consumers can be 
a-h's, so I got an fcc 1st phone without cracking a book and switched to 
broadcasting in the early 60's.  Then in '72 I saw where the local 
community college was testing for the Certified Electronics Technician 
certificate at their site in Norfolk NE.  So I drove the 15 miles down 
from the tv station I was in charge of for Nebraska ETV, a klystron 
powered transmitter on ch19, and laid my 20 bucks on the profs desk at 
12:50, test to start at 13:00. 125 mostly multiple choice, and I handed 
it back to him about 13:40. Accompanied by his raised eyebrows. Told him 
my work history and 2 weeks later rx'd a Journeyman card, I had missed 
2. That card says it all, and I have not failed to get the job I was 
applying for since. That tells the viewer I do know what I'm talking 
about even if he doesn't, which is usually the case.

No, that librarian gets no more support from me. The destruction of that 
much knowledge was criminal in my view.
> Cheers
> -- t


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#219552

[Multipart message — attachments visible in raw view] — view raw

On Thu, Feb 27, 2020 at 08:18:03AM -0500, Gene Heskett wrote:
> On Thursday 27 February 2020 03:50:34 tomas@tuxteam.de wrote:

[...]

> > Alternatively go to your paper library [...]

> This library is a joke, the librarian is scared shitless of copyright 
> law [...]

> No, that librarian gets no more support from me. The destruction of that 
> much knowledge was criminal in my view.

A sad story, for sure. But you might consider giving people a second
chance. Perhaps by now there's some else in charge there.

The Stevens book is highly recommended.

Cheers
-- t

[toc] | [prev] | [next] | [standalone]

#219597

Gene Heskett wrote:
...
> This library is a joke, the librarian is scared shitless of copyright 
> law.  When I retired, I had an 18 year collection of McGraw-Hill's 
> Electronics magazine, from which anybody that could read, could get 
> himself the equ of the best education in electronics available anyplace 
> on the planet, hopeing I might plant a seed in some youngster. Several 
> hundred lbs of it since a year was at least 5-6" on the shelf. I went 
> back to see how they were displayed about a week later, they weren't, 
> the bitch had filed them in the trash bin and they'd already been picked 
> up.  I went home and cry'd. To me, that was the equ of burning the 
> Library of Alexandria.  That magazine was at least half my education in 
> electronics. At that point, in 18 years as CE at WDTV, one piece of gear 
> had been returned to the maker for repair, a Canuk outfit that didn't an 
> american to fix his stuff, I kept everything else ticking along and the 
> case cow fresh.  And you'd be amazed at how quick a news dept. can 
> demolish a $30,000 camera.
>
> Sure, I'm an old fart now, but as a child I had "The Nack" and I quit 
> school over a health issue in the 9th grade, and went to work fixing the 
> then (1948) new-fangled things called televisions, but consumers can be 
> a-h's, so I got an fcc 1st phone without cracking a book and switched to 
> broadcasting in the early 60's.  Then in '72 I saw where the local 
> community college was testing for the Certified Electronics Technician 
> certificate at their site in Norfolk NE.  So I drove the 15 miles down 
> from the tv station I was in charge of for Nebraska ETV, a klystron 
> powered transmitter on ch19, and laid my 20 bucks on the profs desk at 
> 12:50, test to start at 13:00. 125 mostly multiple choice, and I handed 
> it back to him about 13:40. Accompanied by his raised eyebrows. Told him 
> my work history and 2 weeks later rx'd a Journeyman card, I had missed 
> 2. That card says it all, and I have not failed to get the job I was 
> applying for since. That tells the viewer I do know what I'm talking 
> about even if he doesn't, which is usually the case.
>
> No, that librarian gets no more support from me. The destruction of that 
> much knowledge was criminal in my view.

  i support my local library as much as i can, but i
don't visit as often as when i worked there.

  yes, i still miss it, i do not miss the stress.

  i do understand your feelings.  i have a lot of books
here on my shelves that were removed from the reference
collection and the non-fiction collection.  now that the
library has more space i could ask about donating them
back, but my guess is that they too would end up getting
thrown out.


  songbird

[toc] | [prev] | [next] | [standalone]

#219493

On Wednesday 26 February 2020 13:50:40 Lee wrote:

> On 2/26/20, Gene Heskett <gheskett@shentel.net> wrote:
> > over the last 90 days or so, we seem to have been plauged with a new
> > breed of bots scanning our web pages, and they are not just indexing
> > our web pages I don't mind that, but they are ignoring our
> > robots.txt and are  mirroring anything apache2 can reach, including
> > stuff thats there but not reachable by a normal browser just looking
> > around and clicking on links.
>
>   <.. snip ..>
>
> > To add a new rule, covering that whole 256 address block because
> > they seem to have a random address, changed about weekly, in that
> > block:
> >
> > root@coyote:iptables$ cat iptables-add
> >
> > #!/bin/bash
> > iptables -I INPUT -s add.ress.to.block/24 -j DROP
>
> Have you considered REJECT instead of DROP?
>
> REJECT should send a RST telling the other side to give up now.
> DROP just drops the packet leaving the other side to retry until the
> retry limit is hit.
>
> Lee

Thats been considered, but when they go away, they are attacking someone 
else.  And I'm just chrochety enough of an old fart to let them waste 
their time, while saveing my upload bandwidth.


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.debian.user

csiph-web