Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #1484 > unrolled thread

"Timeout" DNS IP to name resolution

Back to article view | Back to comp.os.linux.networking

Contents

  "Timeout" DNS IP to name resolution buck <buck@private.mil> - 2012-06-23 17:56 +0000
    Re: "Timeout" DNS IP to name resolution Chris Davies <chris-usenet@roaima.co.uk> - 2012-06-23 22:29 +0100
      Re: "Timeout" DNS IP to name resolution buck <buck@private.mil> - 2012-06-24 01:05 +0000
        Re: "Timeout" DNS IP to name resolution Pascal Hambourg <boite-a-spam@plouf.fr.eu.org> - 2012-06-24 11:52 +0200
        Re: "Timeout" DNS IP to name resolution Chris Davies <chris-usenet@roaima.co.uk> - 2012-06-24 12:15 +0100
          Re: "Timeout" DNS IP to name resolution Moe Trin <ibuprofin@painkiller.example.tld.invalid> - 2012-06-24 11:09 -0500
            Re: "Timeout" DNS IP to name resolution buck <buck@private.mil> - 2012-06-24 19:50 +0000
              Re: "Timeout" DNS IP to name resolution Moe Trin <ibuprofin@painkiller.example.tld.invalid> - 2012-06-24 19:12 -0500
                Re: "Timeout" DNS IP to name resolution J G Miller <miller@yoyo.ORG> - 2012-06-25 10:22 +0000
    Re: "Timeout" DNS IP to name resolution MJ <me@invalid.org> - 2012-06-27 07:44 +0000
      Re: "Timeout" DNS IP to name resolution buck <buck@private.mil> - 2012-06-29 17:00 +0000
        Re: "Timeout" DNS IP to name resolution Joe Beanfish <joebeanfish@nospam.duh> - 2012-06-29 15:43 -0400
          Re: "Timeout" DNS IP to name resolution buck <buck@private.mil> - 2012-06-30 20:04 +0000
    Re: "Timeout" DNS IP to name resolution Martijn Lievaart <m@rtij.nl.invlalid> - 2012-06-30 00:58 +0200

#1484 — "Timeout" DNS IP to name resolution

I run a rather busy Apache web server with HostnameLookups On because 
management wants it so.

The problem is IP addresses that don't resolve.  There are 2 
possibilities, one of which is that name resolution immediately fails, 
returning a blank/empty set and the other is that the name resolution 
attempt times out.  "Timeout resolution" is causing problems with 
Apache.

My method for working around this is to DROP all requests on port 80 
that originate from IP address blocks that time out...

QUESTIONS
This is difficult for me to explain, so please cut me some slack.  I 
[think|hope] that timeout violates internet standards.  Your help with 
a politely worded and explicit explanation to the network admin, along 
with a description of at least one good way for [him|her] to correct 
the problem, is solicited.

1) What is the difference between immediate return of blank/empty set 
and timeout?

2) Is there an RFC I can read, and perhaps send to the offending 
network administrator, that recommends (or stronger) that IP addresses 
which are not to be resolved return blank?
--
buck

[toc] | [next] | [standalone]

#1485

buck <buck@private.mil> wrote:
> I run a rather busy Apache web server with HostnameLookups On because 
> management wants it so.

I suspect by your phrasing that this isn't a choice you yourself would
have made. Is there any mileage in offering near real-time post-resolution
of names? So Aapche would write IP addresses, and you'd have something
along the lines of this -

    tail -f access.log |
	while read IP OTHERSTUFF
	do
	    NM=`dig +short -x $IP`; echo "${NM:-$IP} $OTHERSTUFF"
	done > access_with_dns.log

(Don't use that code; it's only to illustrate the idea.)


> The problem is IP addresses that don't resolve. [...]

> 1) What is the difference between immediate return of blank/empty set 
> and timeout?

The first is a definitive "there is no information available". But you
can't (shouldn't) draw any definitive conclusions from the second. Maybe
the network was a little slow today. Maybe your UDP packets or the
answers got dropped somewhere en route. Maybe the far end's rDNS really
is misconfigured.

If you're working in a constrained and more controlled situation than
the Internet at large, then maybe some of these assumptions are less
likely to apply.

Chris

[toc] | [prev] | [next] | [standalone]

#1486

Chris Davies <chris-usenet@roaima.co.uk> wrote in
news:v1phb9xvie.ln2@news.roaima.co.uk: 

> buck <buck@private.mil> wrote:
>> I run a rather busy Apache web server with HostnameLookups On 
because
>> management wants it so.
> 
> I suspect by your phrasing that this isn't a choice you yourself 
would
> have made. Is there any mileage in offering near real-time
> post-resolution of names? So Aapche would write IP addresses, and
> you'd have something along the lines of this -
> 
>     tail -f access.log |
>      while read IP OTHERSTUFF
>      do
>          NM=`dig +short -x $IP`; echo "${NM:-$IP} $OTHERSTUFF"
>      done > access_with_dns.log
> 
> (Don't use that code; it's only to illustrate the idea.)
> 
> 
>> The problem is IP addresses that don't resolve. [...]
> 
>> 1) What is the difference between immediate return of blank/empty 
set
>> and timeout?

That will work for some reports.
 
> The first is a definitive "there is no information available". But 
you
> can't (shouldn't) draw any definitive conclusions from the second.
> Maybe the network was a little slow today. Maybe your UDP packets or
> the answers got dropped somewhere en route. Maybe the far end's rDNS
> really is misconfigured.

You say "misconfigured".  Does that mean that rDNS is SUPPOSED to be 
set
up?  If so, are you able to cite a reference. 

FWIW and for an example, any ip in the range 124.114.0.0 thru
124.115.255.255 times out so this is not a "busy" situation. 

Thanks for the pseudo code.
--
buck

[toc] | [prev] | [next] | [standalone]

#1487

Hello,

buck a écrit :
> Chris Davies <chris-usenet@roaima.co.uk> wrote :
> 
>> buck <buck@private.mil> wrote:
>>> 1) What is the difference between immediate return of blank/empty 
>>> set and timeout?
>>  
>> The first is a definitive "there is no information available". But 
>> you can't (shouldn't) draw any definitive conclusions from the second.
>> Maybe the network was a little slow today. Maybe your UDP packets or
>> the answers got dropped somewhere en route. Maybe the far end's rDNS
>> really is misconfigured.
> 
> You say "misconfigured".  Does that mean that rDNS is SUPPOSED to be 
> set up?  If so, are you able to cite a reference. 

Any DNS query should get a reply, either positive or negative, from an
authoritative DNS server.

> FWIW and for an example, any ip in the range 124.114.0.0 thru
> 124.115.255.255 times out so this is not a "busy" situation. 

$ dig -x 124.114.0.0 @ns2.xaonline.com.

; <<>> DiG 9.7.3 <<>> -x 124.114.0.0 @ns2.xaonline.com.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49606
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;0.0.114.124.in-addr.arpa.      IN      PTR

;; ANSWER SECTION:
0.0.114.124.in-addr.arpa. 43200 IN      PTR
0.0.114.124.broad.xa.sn.dynamic.163data.com.cn.

;; AUTHORITY SECTION:
114.124.in-addr.arpa.   43200   IN      NS      ns1.xaonline.com.
114.124.in-addr.arpa.   43200   IN      NS      ns2.xaonline.com.

;; ADDITIONAL SECTION:
ns1.xaonline.com.       86400   IN      A       202.100.4.15
ns2.xaonline.com.       86400   IN      A       61.134.3.11

;; Query time: 516 msec
;; SERVER: 61.134.3.11#53(61.134.3.11)
;; WHEN: Sun Jun 24 11:49:38 2012
;; MSG SIZE  rcvd: 182

$ dig -x 124.114.255.255 @ns1.xaonline.com.

; <<>> DiG 9.7.3 <<>> -x 124.114.255.255 @ns1.xaonline.com.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 41200
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;255.255.114.124.in-addr.arpa.  IN      PTR

;; AUTHORITY SECTION:
114.124.in-addr.arpa.   43200   IN      SOA     localhost.localdomain.
root.localhost.localdomain. 2007022012 10800 3600 604800 86400

;; Query time: 510 msec
;; SERVER: 202.100.4.15#53(202.100.4.15)
;; WHEN: Sun Jun 24 11:49:02 2012
;; MSG SIZE  rcvd: 108

[toc] | [prev] | [next] | [standalone]

#1488

buck <buck@private.mil> wrote:
> You say "misconfigured".  Does that mean that rDNS is SUPPOSED to be 
> set up? If so, are you able to cite a reference. 

I had a quick look through RFCs 1034 and 1035 yesterday evening but
couldn't a definite statement. In your situation I suspect that my
*opinion* is irrelevant, but FWIW I would say that you *should* get some
sort of response for any DNS enquiry, whether forward or reverse.


> FWIW and for an example, any ip in the range 124.114.0.0 thru
> 124.115.255.255 times out so this is not a "busy" situation. 

I get rDNS for 124.114.1.1-254 so maybe this is a peering/routing issue
from your (upstream) provider(s)?

    51.1.114.124.broad.xa.sn.dynamic.163data.com.cn.

Chris

[toc] | [prev] | [next] | [standalone]

#1489

On Sun, 24 Jun 2012, in the Usenet newsgroup comp.os.linux.networking, in
article <0g9jb9xf9.ln2@news.roaima.co.uk>, Chris Davies wrote:

>buck <buck@private.mil> wrote:

>> You say "misconfigured".  Does that mean that rDNS is SUPPOSED to
>> be set up? If so, are you able to cite a reference. 

The DNS RFCs were written before RFC2119 (Key words for use in RFCs to
Indicate Requirement Levels) so there isn't a specific sentence that
says PTR records "MUST" (or even "SHOULD") exist.

>I had a quick look through RFCs 1034 and 1035 yesterday evening but
>couldn't a definite statement.

  RFC 2050    Internet Registry IP Allocation Guidelines  section 5
  RFC 2181    Clarifications to the DNS Specification     section 10.2
  RFC 2317    Classless IN-ADDR.ARPA delegation           entire doc.

none of which are "STANDARD" track documents.

>In your situation I suspect that my *opinion* is irrelevant, but FWIW
>I would say that you *should* get some sort of response for any DNS
>enquiry, whether forward or reverse.

It seems that things fall into three main classes:

  1. That information is SECRET - I don't want you to know it
  2. It's too HARD to set up
  3. What's a reverse zone

This has been an on-going problem for many years - mail administrators
often configure their mail servers to reject mail from systems that
don't resolve, or where the forward and reverse names don't match up.
See question 3.22 in the Sendmail-FAQ as one example.

        Old guy

[toc] | [prev] | [next] | [standalone]

#1490

Moe Trin <ibuprofin@painkiller.example.tld.invalid> wrote in 
news:slrnjueetk.k4l.ibuprofin@fermi.phx.az.us:

> The DNS RFCs were written before RFC2119 (Key words for use in RFCs 
to
> Indicate Requirement Levels) so there isn't a specific sentence that
> says PTR records "MUST" (or even "SHOULD") exist.
> 
>>I had a quick look through RFCs 1034 and 1035 yesterday evening but
>>couldn't a definite statement.
> 
>   RFC 2050    Internet Registry IP Allocation Guidelines  section 5
>   RFC 2181    Clarifications to the DNS Specification     section 
10.2
>   RFC 2317    Classless IN-ADDR.ARPA delegation           entire 
doc.
> 
> none of which are "STANDARD" track documents.
> 
>>In your situation I suspect that my *opinion* is irrelevant, but 
FWIW
>>I would say that you *should* get some sort of response for any DNS
>>enquiry, whether forward or reverse.
> 
> It seems that things fall into three main classes:
> 
>   1. That information is SECRET - I don't want you to know it
>   2. It's too HARD to set up
>   3. What's a reverse zone
> 
> This has been an on-going problem for many years - mail 
administrators
> often configure their mail servers to reject mail from systems that
> don't resolve, or where the forward and reverse names don't match 
up.
> See question 3.22 in the Sendmail-FAQ as one example.

[Moe Trin|Olg Guy]
Thanks.

My reading of 1035 says NO, reverse is not required, but others 
disagree so what do I know?!  All the RFCs I read that talk about 
Email say that rDNS is NOT required, so I presume that the same goes 
for http.

Pascal & Chris,

Thanks.  I changed to a different nameserver and get the expected 
results.

I'm confounded because only 4 netblocks (so far) were timing out, but 
since they no longer do I really don't care to pursue this.
--
buck

[toc] | [prev] | [next] | [standalone]

#1491

On 24 Jun 2012, in the Usenet newsgroup comp.os.linux.networking, in
article <js7r2d02o7k@news6.newsguy.com>, buck wrote:

>Moe Trin <ibuprofin@painkiller.example.tld.invalid> wrote

>> It seems that things fall into three main classes:

>>   1. That information is SECRET - I don't want you to know it
>>   2. It's too HARD to set up
>>   3. What's a reverse zone
 
>> This has been an on-going problem for many years - mail
>> administrators often configure their mail servers to reject mail
>> from systems that don't resolve, or where the forward and reverse
>> names don't match up.    See question 3.22 in the Sendmail-FAQ as
>> one example.

>My reading of 1035 says NO, reverse is not required, but others 
>disagree so what do I know?!

Don't forget that the RFCs are written to create interoperability.
There is no such thing as the Internet Police to enforce them.

>All the RFCs I read that talk about Email say that rDNS is NOT
>required, so I presume that the same goes for http.

Correct - you have to understand that the Internet is a cooperative
grouping of networks. Barring some legal contract between entities,
there is nothing that _requires_ network A to carry traffic to/from
network B.  A frequently heard statement in the now defunct Usenet
newsgroup "news.admin.net-abuse.blocklisting" was "my network - my
rules".  A lot of network abuse was reported from systems and/or
networks that failed to had working or matching name resolution, and
that's why some admins made "proper DNS" as a pre-requisite for
receiving mail (or even just traffic).

        Old guy

[toc] | [prev] | [next] | [standalone]

#1492

On Sunday, June 24th, 2012, at 19:12:43h -0500, Moe Trin wrote:

> Don't forget that the RFCs are written to create interoperability.
> There is no such thing as the Internet Police to enforce them.

But it would appear that in some countries there are police to
control the internet (but not enforce the RFCs).

QUOTE

  November 27th, 2010

  Nominet Plans to Let Police Control UK Internet

  Nominet – the quasi-private entity which controls the .uk part
  of the internet - plans to allow the police to take down any
  website without recourse to the Courts.

  This is at the request of the Serious Organised Crime Agency.
  
UNQUOTE

which has become de facto policy

QUOTE

  UK Domain Seizures: Nominet Admits It's Helped Police Seize 3,000 Sites

  While we've been mostly focused on US-based domain name seizures and attempts
  to expand them using something like COICA, we've also noted that similar issues
  are being discussed in the UK, where Nominet has now admitted that it's helped
  police seize about 3,000 domains based simply upon a request from law enforcement.

  Unlike the US, there isn't even a formal process with a judge rubber stamping
  the requests. Instead, the police ask, and Nominet is compelled to suspend the domain.

  In fact, some law enforcement officials are claiming that if Nominet refused their
  requests, then it would automatically become liable. In other words, police have
  a fantastic tool for censorship of any website if they want to use it that way. 

UNQUOTE

Welcome to the Internet controlled police state.

[toc] | [prev] | [next] | [standalone]

#1495

On Sat, 23 Jun 2012 17:56:22 +0000, buck wrote in
js5006016b1@news1.newsguy.com :

> I run a rather busy Apache web server with HostnameLookups On because
> management wants it so.
> 
> The problem is IP addresses that don't resolve. ...

Management would be well-advised to allow you to use the Perl script 
named "jdresolve" to resolve IP addresses whenever any logfile analyses 
are needed, or subsets thereof. It can quickly recurse through any 
addresses to resolve domains for which A or PTR records don't exist (such 
as your point of failure), e.g.:

$ echo 192.168.1.254 | jdresolve -n -a -r -
192.168.1.254.starband.com
$ echo 123.45.67.89 | jdresolve -n -a -r -
123.45.67.89.dns.kr

or entire files:

# jdresolve -r -n -a /var/log/apache.log > /tmp/resolved.apache.log

Your management will be highly pleased with your creative shell-scripted 
uses of the program, I promise. 

An example "probe_report" script output of dropped, logged, unsolicited 
packets from iptables:

# /usr/local/bin/probe_report

Resolved_Address              Packets Bytes  Protocol(s) Dest.Port(s)
195.161.40.7.rt-comm.ru       1       28     ICMP
211.147.3.19.datadragon.net   1       44     TCP         22
acyv254.neoplus.adsl.tpnet.pl 1       44     TCP         110
192.168.1.99.starband.com     210     82600  UDP         631
59.108.123.83.in-addr.cn      1       44     TCP         8080
_____________________________________________________________
Totals                        214     80.8KB for search pattern 
"UNSOLICITED"

[toc] | [prev] | [next] | [standalone]

#1499

MJ <me@invalid.org> wrote in news:a4vrv9F21bU2@mid.individual.net:

> On Sat, 23 Jun 2012 17:56:22 +0000, buck wrote in
> js5006016b1@news1.newsguy.com :
> 
>> I run a rather busy Apache web server with HostnameLookups On 
because
>> management wants it so.
>> 
>> The problem is IP addresses that don't resolve. ...
> 
> Management would be well-advised to allow you to use the Perl script 
> named "jdresolve" to resolve IP addresses whenever any logfile
> analyses are needed, or subsets thereof. It can quickly recurse
> through any addresses to resolve domains for which A or PTR records
> don't exist (such as your point of failure), e.g.:

jdresolve only resolves 60% of the IPs encountered by my web server
yesterday.  That's not good enough,
--
buck

[toc] | [prev] | [next] | [standalone]

#1500

On 06/29/2012 01:00 PM, buck wrote:
> MJ <me@invalid.org> wrote in news:a4vrv9F21bU2@mid.individual.net:
>
>> On Sat, 23 Jun 2012 17:56:22 +0000, buck wrote in
>> js5006016b1@news1.newsguy.com :
>>
>>> I run a rather busy Apache web server with HostnameLookups On
> because
>>> management wants it so.
>>>
>>> The problem is IP addresses that don't resolve. ...
>>
>> Management would be well-advised to allow you to use the Perl script
>> named "jdresolve" to resolve IP addresses whenever any logfile
>> analyses are needed, or subsets thereof. It can quickly recurse
>> through any addresses to resolve domains for which A or PTR records
>> don't exist (such as your point of failure), e.g.:
>
> jdresolve only resolves 60% of the IPs encountered by my web server
> yesterday.  That's not good enough,
> --
> buck
>

Not all IPs have names. Those will never resolve.
Question is whether jdresolve resolves any fewer than apache?

[toc] | [prev] | [next] | [standalone]

#1505

Joe Beanfish <joebeanfish@nospam.duh> wrote in news:jsl0hn$q89$1@dont-
email.me:

> Not all IPs have names. Those will never resolve.
> Question is whether jdresolve resolves any fewer than apache?

Since Apache is now happy, I just reported my jdresolve "60% problem" 
for those who stumble across this thread.

Apache does a lot better than jdresolve at rDNS.  My upstream has 3 
servers, 2 of which I'll call "balky" because they sometimes work and 
sometimes/often fail.  Since I set up the system to use the one server 
that always works, the problem has been alleviated.  Now, only a few 
timeouts a day occur.

I think the problem with jdresolve is the Net::DNS Perl module it 
requires.  If it were to become necessary, I could get the jdresolve 
script worked on to become hierarchical, first trying a caching DNS on 
the LAN, then the upstream NS, and finally - as a "last resort" effort 
- the root servers.  But that isn't likely because the only benefits I 
find in jdresolve are that the IP can be anywhere in the logged line 
and that recursion can be turned on.  Neither of those things are 
presently necessary.
--
buck

[toc] | [prev] | [next] | [standalone]

#1502

On Sat, 23 Jun 2012 17:56:22 +0000, buck wrote:

> I run a rather busy Apache web server with HostnameLookups On because
> management wants it so.

Tell management that if they really want this, they need something like  
10 times the amount of webservers to handle the same amount of requests. 
And even then, there are no guarentees, so 20 times the amount of 
webservers would be even better. If they are willing to invest this kind 
of money fine. If not, point them to the alternatives given in this 
thread.

rDNS requests can and will time out for various reasons, legitimate or 
not. It's easier to resolve afterwards. This may mean that you see some 
rDNSses that are not the same as at the moment the request came in to 
your webserver, but hey, rDNS is informative anyhow. People can set it up 
to whatever they like, so it cannot be relied on anyhow.

Another idea would be to logrotate-post-process the logs to do the rDNS 
then. Maybe that's acceptable?

M4

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.networking

csiph-web