Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #4673 > unrolled thread

Delay when SSH'ing into some systems in my LAN

Back to article view | Back to comp.os.linux.networking

Contents

  Delay when SSH'ing into some systems in my LAN "S.K.R. de Jong" <SKRdJ@nowhere.net> - 2020-07-21 15:27 +0000
    Re: Delay when SSH'ing into some systems in my LAN William Unruh <unruh@invalid.ca> - 2020-07-21 16:28 +0000
      Re: Delay when SSH'ing into some systems in my LAN "S.K.R. de Jong" <SKRdJ@nowhere.net> - 2020-07-21 18:29 +0000
        Re: Delay when SSH'ing into some systems in my LAN William Unruh <unruh@invalid.ca> - 2020-07-21 19:52 +0000
      Re: Delay when SSH'ing into some systems in my LAN "S.K.R. de Jong" <SKRdJ@nowhere.net> - 2020-07-21 19:33 +0000

#4673 — Delay when SSH'ing into some systems in my LAN

	This started happening recently, with no apparent reason.

	I have systems A, B, C, D in my LAN. When I SSH from A into B the 
connection gets established immediately. However when I do it from A to C 
I get the following SSH traces in A:

	OpenSSH_7.4p1, OpenSSL 1.0.2u  20 Dec 2019
	debug1: Reading configuration data /home/xyz/.ssh/config
	debug1: Reading configuration data /etc/ssh/ssh_config
	debug2: resolving "C" port 22

It hangs at that point for 20 seconds or so, but then the connection 
succeeds. Once it is established, the performance of the session is as 
expected. The same behavior is observed when I try to SSH from A into D.

	Taking the "resolving "C" port 22" as a clue, I thought that I 
might be having DNS problems in A. That does not seem to be the case 
though:

	$ dig C

	; <<>> DiG 9.11.19 <<>> mimir
	;; global options: +cmd
	;; Got answer:
	;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63806
	;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, 
ADDITIONAL: 1

	;; OPT PSEUDOSECTION:
	; EDNS: version: 0, flags:; udp: 4096
	;; QUESTION SECTION:
	;C.				IN	A

	;; ANSWER SECTION:
	C.			0	IN	A	192.168.0.9

	;; Query time: 0 msec
	;; SERVER: 192.168.0.7#53(192.168.0.7)
	;; WHEN: Tue Jul 21 09:07:02 MDT 2020
	;; MSG SIZE  rcvd: 50

The resolution happens with no delay, but the SSH connection attempt to C 
keeps taking 20 seconds, as above. And analogously for D.

	Now once I have SSH'd into C, I can SSH from C into D with no 
delay. I can even SSH from C into A (or B, or D) with no delay 
whatsoever. In general, if A is not involved as a client things seem to 
be normal.

	So it would seem that the problem is in A alone, and not when 
trying to SSH into every system - only some seem to be affected.

	Any suggestions on how to diagnose this? I can't see anything 
relevant in the logs of the systems involved - the only thing present is 
the delay I mentioned, in the circumstances that I described. In A I have 
the following IPtables rules:

	Chain INPUT (policy ACCEPT)
	target     prot opt source               destination         

	Chain FORWARD (policy DROP)
	target     prot opt source               destination         
	ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            	
state RELATED,ESTABLISHED
	ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           

	Chain OUTPUT (policy ACCEPT)
	target     prot opt source               destination         

whereas in the other systems I have no IPtables rules. I stopped and 
starting the network in A, to no avail. I guess that rebooting A might 
solve the problem. Or not. Even if it does, I would not understand why.

[toc] | [next] | [standalone]

#4674

On 2020-07-21, S.K.R. de Jong <SKRdJ@nowhere.net> wrote:
> 	This started happening recently, with no apparent reason.
>
> 	I have systems A, B, C, D in my LAN. When I SSH from A into B the 
> connection gets established immediately. However when I do it from A to C 
> I get the following SSH traces in A:
>
> 	OpenSSH_7.4p1, OpenSSL 1.0.2u  20 Dec 2019
> 	debug1: Reading configuration data /home/xyz/.ssh/config
> 	debug1: Reading configuration data /etc/ssh/ssh_config
> 	debug2: resolving "C" port 22

Try to ssh into C using C's IP address rather than name. 
If that goes immediately
Try putting C's name into A's /etc/hosts
Since it seems to be A that is complaining, try putting C's name and
address into /etc/hosts on A. 
It seem's that it is the reverse dns that is problematic.
Note that in the above A,B,C,D could be names or they could be IP
addresses. It might help to actually give us the exact log lines, rather
than the edited ones.


>
> It hangs at that point for 20 seconds or so, but then the connection 
> succeeds. Once it is established, the performance of the session is as 
> expected. The same behavior is observed when I try to SSH from A into D.
>
> 	Taking the "resolving "C" port 22" as a clue, I thought that I 
> might be having DNS problems in A. That does not seem to be the case 
> though:
>
> 	$ dig C
>
> 	; <<>> DiG 9.11.19 <<>> mimir
> 	;; global options: +cmd
> 	;; Got answer:
> 	;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63806
> 	;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, 
> ADDITIONAL: 1
>
> 	;; OPT PSEUDOSECTION:
> 	; EDNS: version: 0, flags:; udp: 4096
> 	;; QUESTION SECTION:
> 	;C.				IN	A
>
> 	;; ANSWER SECTION:
> 	C.			0	IN	A	192.168.0.9
>
> 	;; Query time: 0 msec
> 	;; SERVER: 192.168.0.7#53(192.168.0.7)
> 	;; WHEN: Tue Jul 21 09:07:02 MDT 2020
> 	;; MSG SIZE  rcvd: 50
>
> The resolution happens with no delay, but the SSH connection attempt to C 
> keeps taking 20 seconds, as above. And analogously for D.
>
> 	Now once I have SSH'd into C, I can SSH from C into D with no 
> delay. I can even SSH from C into A (or B, or D) with no delay 
> whatsoever. In general, if A is not involved as a client things seem to 
> be normal.
>
> 	So it would seem that the problem is in A alone, and not when 
> trying to SSH into every system - only some seem to be affected.
>
> 	Any suggestions on how to diagnose this? I can't see anything 
> relevant in the logs of the systems involved - the only thing present is 
> the delay I mentioned, in the circumstances that I described. In A I have 
> the following IPtables rules:
>
> 	Chain INPUT (policy ACCEPT)
> 	target     prot opt source               destination         
>
> 	Chain FORWARD (policy DROP)
> 	target     prot opt source               destination         
> 	ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            	
> state RELATED,ESTABLISHED
> 	ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
>
> 	Chain OUTPUT (policy ACCEPT)
> 	target     prot opt source               destination         
>
> whereas in the other systems I have no IPtables rules. I stopped and 
> starting the network in A, to no avail. I guess that rebooting A might 
> solve the problem. Or not. Even if it does, I would not understand why.

[toc] | [prev] | [next] | [standalone]

#4675

On Tue, 21 Jul 2020 16:28:57 +0000, William Unruh wrote:

> On 2020-07-21, S.K.R. de Jong <SKRdJ@nowhere.net> wrote:
>> 	This started happening recently, with no apparent reason.
>>
>> 	I have systems A, B, C, D in my LAN. When I SSH from A into B the
>> connection gets established immediately. However when I do it from A to
>> C I get the following SSH traces in A:
>>
>> 	OpenSSH_7.4p1, OpenSSL 1.0.2u  20 Dec 2019 debug1: Reading
>> 	configuration data /home/xyz/.ssh/config debug1: Reading 
configuration
>> 	data /etc/ssh/ssh_config debug2: resolving "C" port 22
> 
> Try to ssh into C using C's IP address rather than name.
> If that goes immediately Try putting C's name into A's /etc/hosts Since
> it seems to be A that is complaining, try putting C's name and address
> into /etc/hosts on A.

	Thanks. When using IP addresses instead, or entering the relevant 
names into /etc/hosts, the delay disappears.

> It seem's that it is the reverse dns that is problematic.

	I would agree.

> Note that in the above A,B,C,D could be names or they could be IP
> addresses. It might help to actually give us the exact log lines, rather
> than the edited ones.

	They were meant to be names, not IP addresses.

[toc] | [prev] | [next] | [standalone]

#4677

On 2020-07-21, S.K.R. de Jong <SKRdJ@nowhere.net> wrote:
> On Tue, 21 Jul 2020 16:28:57 +0000, William Unruh wrote:
>
>> On 2020-07-21, S.K.R. de Jong <SKRdJ@nowhere.net> wrote:
>>> 	This started happening recently, with no apparent reason.
>>>
>>> 	I have systems A, B, C, D in my LAN. When I SSH from A into B the
>>> connection gets established immediately. However when I do it from A to
>>> C I get the following SSH traces in A:
>>>
>>> 	OpenSSH_7.4p1, OpenSSL 1.0.2u  20 Dec 2019 debug1: Reading
>>> 	configuration data /home/xyz/.ssh/config debug1: Reading 
> configuration
>>> 	data /etc/ssh/ssh_config debug2: resolving "C" port 22
>> 
>> Try to ssh into C using C's IP address rather than name.
>> If that goes immediately Try putting C's name into A's /etc/hosts Since
>> it seems to be A that is complaining, try putting C's name and address
>> into /etc/hosts on A.
>
> 	Thanks. When using IP addresses instead, or entering the relevant 
> names into /etc/hosts, the delay disappears.
>
>> It seem's that it is the reverse dns that is problematic.
>
> 	I would agree.
>
>> Note that in the above A,B,C,D could be names or they could be IP
>> addresses. It might help to actually give us the exact log lines, rather
>> than the edited ones.
>
> 	They were meant to be names, not IP addresses.

So you are running an internal dns server? If you have your machines on
fixed addresses, and do not have too many of them, using /etc/hosts is
probably more efficient. If they are using dhcp and the various machine
have constantly fluctuating IP addresses, or you have lots of
machines(lets say>10) then a dns server is probably best and you then
need to figure out why it is not delivering on a dns query. tcpdump on a
third machine on that network would show if the query was actually being
sent out to the dns server, and whether it is returning reasonable
answers of grabage. Also, it would seem that you have a secondary dns
server which is used after the timeout on the first one. I assume it is
also local-- so why is it returning something reasonable and the first
is not. 
>

[toc] | [prev] | [next] | [standalone]

#4676

On Tue, 21 Jul 2020 16:28:57 +0000, William Unruh wrote:

> On 2020-07-21, S.K.R. de Jong <SKRdJ@nowhere.net> wrote:
>> 	This started happening recently, with no apparent reason.
>>
>> 	I have systems A, B, C, D in my LAN. When I SSH from A into B the
>> connection gets established immediately. However when I do it from A to
>> C I get the following SSH traces in A:
>>
>> 	OpenSSH_7.4p1, OpenSSL 1.0.2u  20 Dec 2019 debug1: Reading
>> 	configuration data /home/xyz/.ssh/config debug1: Reading 
configuration
>> 	data /etc/ssh/ssh_config debug2: resolving "C" port 22
> 
> Try to ssh into C using C's IP address rather than name.
> If that goes immediately Try putting C's name into A's /etc/hosts Since
> it seems to be A that is complaining, try putting C's name and address
> into /etc/hosts on A.
> It seem's that it is the reverse dns that is problematic.
> Note that in the above A,B,C,D could be names or they could be IP
> addresses. It might help to actually give us the exact log lines, rather
> than the edited ones.

	Well, I found the solution, but I am not entirely sure why it was 
broken in the first place.

	In A, the contents of my /etc/resolv.conf have the following line 
at the top:

	search <my-domain> <work-domain>

where <mydomain> is a domain assigned to my LAN, and <work-domain> is the 
domain of my employers, whose systems I access over a VPN.

	When I do

	ssh B

the connection is immediately sent to system B in my LAN, and everything 
is fine - no delay at all. But when trying the same with C, the DNS 
server first seems to try to resolve C.<work-domain> - which is the 
origin of the delay. I don't know why it does so for C (and D) but not 
for B, but the fact is that removing <work-domain> from the search line 
in A's /etc/resolv.conf file solves the problem.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.networking

csiph-web