Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.user > #208667
| From | Greg Wooledge <wooledg@eeg.ccf.org> |
|---|---|
| Newsgroups | linux.debian.user |
| Subject | Re: baido.com |
| Date | 2019-05-15 15:40 +0200 |
| Message-ID | <xY2it-5YW-7@gated-at.bofh.it> (permalink) |
| References | <xY28N-5VJ-1@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
On Wed, May 15, 2019 at 09:29:15AM -0400, Gene Heskett wrote: > Greetings all; > > I just caught the baido-spider crawling my site for about 15 minutes. > > Is putting www.baido.com in my hosts.deny enough to shut that down? Or > is there a deny function in apache2 I should be using instead? https://www.google.com/search?q=robots.txt+baido In general, use the robots.txt file in your web server first and foremost. If that doesn't work, THEN move on to heavier solutions like firewalls.
Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
baido.com Gene Heskett <gheskett@shentel.net> - 2019-05-15 15:30 +0200
Re: baido.com Greg Wooledge <wooledg@eeg.ccf.org> - 2019-05-15 15:40 +0200
Re: baido.com Brent Clark <brentgclarklist@gmail.com> - 2019-05-15 15:50 +0200
Re: baido.com Gene Heskett <gheskett@shentel.net> - 2019-05-15 17:50 +0200
csiph-web