Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #7239 > unrolled thread

problem with link checker

Back to article view | Back to comp.lang.php

Contents

  problem with link checker Helmut <helmut_blass@web.de> - 2012-10-03 02:46 -0700
    Re: problem with link checker Captain Paralytic <paul_lautman@yahoo.com> - 2012-10-03 02:48 -0700
      Re: problem with link checker Jerry Stuckle <jstucklex@attglobal.net> - 2012-10-03 11:20 -0400
    Re: problem with link checker "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2012-10-03 13:34 +0200
      Re: problem with link checker Jerry Stuckle <jstucklex@attglobal.net> - 2012-10-03 11:21 -0400
    Re: problem with link checker Martin Leese <please@see.Web.for.e-mail.INVALID> - 2012-10-03 08:07 -0600

#7239 — problem with link checker

Hello,
I have written a php script whose function is to detect dead links on 
a website.
unfortunately,(obviously) my ISP  redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
Even worse, in one case an invalid url (http://www.ajrccm.com)
is also redirected to a search site (I don't know by whom) and I get
the status 200 !
Any idea how to solve this problem?

thanx for your help, Helmut

[toc] | [next] | [standalone]

#7240

On Oct 3, 10:46 am, Helmut <helmut_bl...@web.de> wrote:
> Hello,
> I have written a php script whose function is to detect dead links on
> a website.
> unfortunately,(obviously) my ISP  redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
> Even worse, in one case an invalid url (http://www.ajrccm.com)
> is also redirected to a search site (I don't know by whom) and I get
> the status 200 !
> Any idea how to solve this problem?

Change your ISP! Alternatively, my ISP also has this facility, but
offers a way to switch if off if you don't wish to use it.

[toc] | [prev] | [next] | [standalone]

#7244

On 10/3/2012 5:48 AM, Captain Paralytic wrote:
> On Oct 3, 10:46 am, Helmut <helmut_bl...@web.de> wrote:
>> Hello,
>> I have written a php script whose function is to detect dead links on
>> a website.
>> unfortunately,(obviously) my ISP  redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
>> Even worse, in one case an invalid url (http://www.ajrccm.com)
>> is also redirected to a search site (I don't know by whom) and I get
>> the status 200 !
>> Any idea how to solve this problem?
>
> Change your ISP! Alternatively, my ISP also has this facility, but
> offers a way to switch if off if you don't wish to use it.
>

Hi, Paul,

Unfortunately, that isn't always possible - or prudent.

I have the same problem here, with no way to turn it off.  There's only 
one other ISP here, and we switched from them about 3 years ago because 
of their poor service (internet was out for a week because of a "line 
problem" - and not the first time!).

I hadn't thought about Matt's suggestion to change to a different DNS 
because it hasn't been that much of a problem (I just do link checking 
by SSHing into the server).  But it's been a hassle at times (i.e. type 
in a long URL but mistype one character - and have to enter it all over 
again) that I'd like to cure.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#7241

Am 03.10.2012 11:46, schrieb Helmut:
> Hello,
> I have written a php script whose function is to detect dead links on 
> a website.
> unfortunately,(obviously) my ISP  redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
> Even worse, in one case an invalid url (http://www.ajrccm.com)
> is also redirected to a search site (I don't know by whom) and I get
> the status 200 !
> Any idea how to solve this problem?
> 
> thanx for your help, Helmut
> 

1. Find a name server that does not spam DNS, see terminal log below.
2. on Linux I would not hesitate to install bind9, it's a one liner

/Str.
#
# host not found, okay
#
strobel@s114-intel:~> ping nonono.deee
ping: unknown host nonono.deee
#
# dig @ google nameserver, not found, okay
#
strobel@s114-intel:~> dig nonono.deee @8.8.8.8

; <<>> DiG 9.7.6-P3 <<>> nonono.deee @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 14380
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;nonono.deee.                   IN      A

;; AUTHORITY SECTION:
.                       1800    IN      SOA     a.root-servers.net.
nstld.verisign-grs.com. 2012100300 1800 900 604800 86400

;; Query time: 92 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Oct  3 13:27:20 2012
;; MSG SIZE  rcvd: 104

#
# dig @ dnsadvantage, Spam!
#
strobel@s114-intel:~> dig nonono.deee @156.154.70.1

; <<>> DiG 9.7.6-P3 <<>> nonono.deee @156.154.70.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 644
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;nonono.deee.                   IN      A

;; ANSWER SECTION:
nonono.deee.            600     IN      A       92.242.144.2

;; Query time: 66 msec
;; SERVER: 156.154.70.1#53(156.154.70.1)
;; WHEN: Wed Oct  3 13:27:52 2012
;; MSG SIZE  rcvd: 56

strobel@s114-intel:~>

[toc] | [prev] | [next] | [standalone]

#7245

On 10/3/2012 7:34 AM, M. Strobel wrote:
> Am 03.10.2012 11:46, schrieb Helmut:
>> Hello,
>> I have written a php script whose function is to detect dead links on
>> a website.
>> unfortunately,(obviously) my ISP  redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
>> Even worse, in one case an invalid url (http://www.ajrccm.com)
>> is also redirected to a search site (I don't know by whom) and I get
>> the status 200 !
>> Any idea how to solve this problem?
>>
>> thanx for your help, Helmut
>>
>
> 1. Find a name server that does not spam DNS, see terminal log below.
> 2. on Linux I would not hesitate to install bind9, it's a one liner
>
> /Str.

Hmmm, I have the same problem here but it's more of a hassle than a 
problem (see my note to Paul).  Hadn't thought about changing name 
servers.  Might have to do that.  I've already got bind9 on my servers.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#7242

Helmut wrote:
> Hello,
> I have written a php script whose function is to detect dead links on 
> a website.
> unfortunately,(obviously) my ISP  redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
> Even worse, in one case an invalid url (http://www.ajrccm.com)
> is also redirected to a search site (I don't know by whom) and I get
> the status 200 !
> Any idea how to solve this problem?

Use http://validator.w3.org/checklink

This one is written in Perl.

-- 
Regards,
Martin Leese
E-mail: please@see.Web.for.e-mail.INVALID
Web: http://members.tripod.com/martin_leese/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.php

csiph-web