Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #3205 > unrolled thread

Strange ARP flood

Back to article view | Back to comp.os.linux.networking

Contents

  Strange ARP flood pk <pk@pk.invalid> - 2015-01-13 18:56 +0100
    Re: Strange ARP flood Aleksandar Kuktin <akuktin@gmail.com> - 2015-01-13 19:44 +0000
      Re: Strange ARP flood pk <pk@pk.invalid> - 2015-01-14 10:26 +0100
    Re: Strange ARP flood detha <detha@foad.co.za> - 2015-01-14 21:36 +0200
      Re: Strange ARP flood pk <pk@pk.invalid> - 2015-01-16 10:03 +0100
        Re: Strange ARP flood detha <detha@foad.co.za> - 2015-01-17 08:15 +0200
          Re: Strange ARP flood pk <pk@pk.invalid> - 2015-01-19 11:13 +0100
            Re: Strange ARP flood Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2015-01-19 14:13 +0200
              Re: Strange ARP flood pk <pk@pk.invalid> - 2015-01-19 13:34 +0100
    Re: Strange ARP flood Ralph Spitzner <rasp@spitzner.org> - 2015-01-15 10:12 +0100
      Re: Strange ARP flood pk <pk@pk.invalid> - 2015-01-16 10:01 +0100

#3205 — Strange ARP flood

(Asking here for lack of a better newsgroup; direct me as appropriate if
I'm wrong)

The network is a mixture of physical ethernet switches and wireless AP,
everything is in the same broadcast domain (please no comments about this).

From time to time, we see a flood of strange ARP frames, every time it's a
single machine that's flooding, for example like this:

18:23:44.326912 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
18:23:44.326967 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
18:23:44.327021 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
...

In this case, the machine at 3c:15:c2:c8:cb:11 happens to be an Apple
laptop connected to the wifi, which indeed has (already) the IP address
10.15.10.251. Tearing down the wireless connection end re-enabling it makes
the traffic disappear.

A quick audit of the laptop while the flood is happening does not reveal
any special or strange application running.

The same happens from time to time to other devices (the great majority of
times they were Apple laptopos, but we've seen other devices as well,
namely a few mobile phones). They're always devices connected to the wifi;
we've never seen the flood coming from wired devices.

Any ideas what could be causing this?

[toc] | [next] | [standalone]

#3206

On Tue, 13 Jan 2015 18:56:26 +0100, pk wrote:

> (Asking here for lack of a better newsgroup; direct me as appropriate if
> I'm wrong)
> 
> The network is a mixture of physical ethernet switches and wireless AP,
> everything is in the same broadcast domain (please no comments about
> this).
> 
> From time to time, we see a flood of strange ARP frames, every time it's
> a single machine that's flooding, for example like this:
> 
> 18:23:44.326912 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
> 18:23:44.326967 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
> 18:23:44.327021 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46 ...
> 
> In this case, the machine at 3c:15:c2:c8:cb:11 happens to be an Apple
> laptop connected to the wifi, which indeed has (already) the IP address
> 10.15.10.251. Tearing down the wireless connection end re-enabling it
> makes the traffic disappear.
> 
> A quick audit of the laptop while the flood is happening does not reveal
> any special or strange application running.
> 
> The same happens from time to time to other devices (the great majority
> of times they were Apple laptopos, but we've seen other devices as well,
> namely a few mobile phones). They're always devices connected to the
> wifi;
> we've never seen the flood coming from wired devices.
> 
> Any ideas what could be causing this?

Triplets of ARP similar to what you describe (but with a larger 
interpacket gap, I think) are used by Windows to check the availability 
of an IP address before binding to it.

Missing from your description is a more detailed account of the flood. 
How many packets in a flood? Time delta between packets in a flood, 
distance in time between two floods and so on.

It could just be that the host is trying to bind to an IP address but has 
problems with it. This can be tested by looking at address timeouts given 
by DHCP and trying to correlate them with floods.

[toc] | [prev] | [next] | [standalone]

#3207

On Tue, 13 Jan 2015 19:44:33 +0000 (UTC), Aleksandar Kuktin
<akuktin@gmail.com> wrote:

> Triplets of ARP similar to what you describe (but with a larger 
> interpacket gap, I think) are used by Windows to check the availability 
> of an IP address before binding to it.
> 
> Missing from your description is a more detailed account of the flood. 
> How many packets in a flood? Time delta between packets in a flood, 
> distance in time between two floods and so on.

Well, it's a continuous, steady flood. I happened to paste just three
packets as an example, but they're really continuous. They happen in the
order of ~5000 frames per second (give or take), as can be seen from the
deltas of the three I pasted.

As for the distance between two floods, so far it's been unpredictable; at
some point, we just get traffic alerts from the monitoring system, at which
point a quick capture confirms that we're getting the ARP flood, and
proceed to inform the (unaware) offending user, who then resets his wireless
connection and the flood stops.
 
> It could just be that the host is trying to bind to an IP address but has 
> problems with it. This can be tested by looking at address timeouts given 
> by DHCP and trying to correlate them with floods.

Will check that, thanks.

[toc] | [prev] | [next] | [standalone]

#3208

On Tue, 13 Jan 2015 18:56:26 +0100, pk wrote:

> (Asking here for lack of a better newsgroup; direct me as appropriate if
> I'm wrong)
> 
> The network is a mixture of physical ethernet switches and wireless AP,
> everything is in the same broadcast domain (please no comments about
> this).
> 
> From time to time, we see a flood of strange ARP frames, every time it's a
> single machine that's flooding, for example like this:
> 
> 18:23:44.326912 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
> 18:23:44.326967 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
> 18:23:44.327021 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46 ...
> 
> In this case, the machine at 3c:15:c2:c8:cb:11 happens to be an Apple
> laptop connected to the wifi, which indeed has (already) the IP address
> 10.15.10.251. Tearing down the wireless connection end re-enabling it
> makes the traffic disappear.

How sure are you that there are no loops in the topology?

Are you running (R)STP ?

-d

[toc] | [prev] | [next] | [standalone]

#3212

On Wed, 14 Jan 2015 21:36:33 +0200, detha <detha@foad.co.za> wrote:

> On Tue, 13 Jan 2015 18:56:26 +0100, pk wrote:
> 
> > (Asking here for lack of a better newsgroup; direct me as appropriate if
> > I'm wrong)
> > 
> > The network is a mixture of physical ethernet switches and wireless AP,
> > everything is in the same broadcast domain (please no comments about
> > this).
> > 
> > From time to time, we see a flood of strange ARP frames, every time
> > it's a single machine that's flooding, for example like this:
> > 
> > 18:23:44.326912 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> > (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> > 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
> > 18:23:44.326967 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> > (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> > 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
> > 18:23:44.327021 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> > (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> > 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46 ...
> > 
> > In this case, the machine at 3c:15:c2:c8:cb:11 happens to be an Apple
> > laptop connected to the wifi, which indeed has (already) the IP address
> > 10.15.10.251. Tearing down the wireless connection end re-enabling it
> > makes the traffic disappear.
> 
> How sure are you that there are no loops in the topology?

No loops.
 
> Are you running (R)STP ?

I'm running RSTP. It doesn't look like an STP loop thing, since if I
isolate the offending machine from the network the flood stops. If it were
a topology/loop problem, it wouldn't stop unless the loop would be
physically broken.

[toc] | [prev] | [next] | [standalone]

#3215

On Fri, 16 Jan 2015 10:03:58 +0100, pk wrote:

> On Wed, 14 Jan 2015 21:36:33 +0200, detha <detha@foad.co.za> wrote:
> 
>> On Tue, 13 Jan 2015 18:56:26 +0100, pk wrote:

>> > (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
>> > 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46 ...
>> > 
>> > In this case, the machine at 3c:15:c2:c8:cb:11 happens to be an Apple
>> > laptop connected to the wifi, which indeed has (already) the IP
>> > address 10.15.10.251. Tearing down the wireless connection end
>> > re-enabling it makes the traffic disappear.
>> 
>> How sure are you that there are no loops in the topology?
> 
> No loops.
>  
>> Are you running (R)STP ?
> 
> I'm running RSTP. It doesn't look like an STP loop thing, since if I
> isolate the offending machine from the network the flood stops. If it were
> a topology/loop problem, it wouldn't stop unless the loop would be
> physically broken.

Correct. However it looks like some form of gratuitous ARP, amplified
through a broadcast loop. Could it be a buggy network stack that triggers
some interaction between the client and the AP that re-broadcasts it?

Which leads to: 'Does it always happen with a particular client, or with a
particular client OS?' and 'Does it always happen when the client is
connected to a particular AP, or to a particular brand AP?'

-d

[toc] | [prev] | [next] | [standalone]

#3220

On Sat, 17 Jan 2015 08:15:13 +0200, detha <detha@foad.co.za> wrote:

> >> Are you running (R)STP ?
> > 
> > I'm running RSTP. It doesn't look like an STP loop thing, since if I
> > isolate the offending machine from the network the flood stops. If it
> > were a topology/loop problem, it wouldn't stop unless the loop would be
> > physically broken.
> 
> Correct. However it looks like some form of gratuitous ARP, amplified
> through a broadcast loop. Could it be a buggy network stack that triggers
> some interaction between the client and the AP that re-broadcasts it?

Uhm, AFAIK gratuitous ARPs are essentially ARP replies, while here we're
seeing ARP requests.

However, I certainly can't exclude a buggy OS, especially since we're
talking Apple here (see below).

 
> Which leads to: 'Does it always happen with a particular client, or with a
> particular client OS?' and 'Does it always happen when the client is
> connected to a particular AP, or to a particular brand AP?'

So far all the cases have been wirelessly connected Apple laptops,
connected to different APs (so it doesn't look like something specific of
a particular AP).

At this point, next step will be detailed analysis of an offending machine
while the flood is happening.

Thanks for your suggestions.

[toc] | [prev] | [next] | [standalone]

#3221

On 19.1.15 12:13, pk wrote:
> On Sat, 17 Jan 2015 08:15:13 +0200, detha <detha@foad.co.za> wrote:
>
>>>> Are you running (R)STP ?
>>>
>>> I'm running RSTP. It doesn't look like an STP loop thing, since if I
>>> isolate the offending machine from the network the flood stops. If it
>>> were a topology/loop problem, it wouldn't stop unless the loop would be
>>> physically broken.
>>
>> Correct. However it looks like some form of gratuitous ARP, amplified
>> through a broadcast loop. Could it be a buggy network stack that triggers
>> some interaction between the client and the AP that re-broadcasts it?
>
> Uhm, AFAIK gratuitous ARPs are essentially ARP replies, while here we're
> seeing ARP requests.
>
> However, I certainly can't exclude a buggy OS, especially since we're
> talking Apple here (see below).


No - GARP is a request for own IP [1]. The idea is to invite response
from some other host having the same IP, to avoid collisions.

The GARP's are probably belonging to Apple Rendezvous (Zeroconf)
protocol looking for an address to use, when the host has no other
useful address. Are they for 169.254.x.y?

[1] W. Richard Stevens, TCP/IP Illustrated, Volume 1 The Protocols,
     ISBN 0-201-63346-9, pp. 62-63

-- 

Tauno Voipio

[toc] | [prev] | [next] | [standalone]

#3222

On Mon, 19 Jan 2015 14:13:54 +0200, Tauno Voipio
<tauno.voipio@notused.fi.invalid> wrote:

> On 19.1.15 12:13, pk wrote:
> > On Sat, 17 Jan 2015 08:15:13 +0200, detha <detha@foad.co.za> wrote:
> >
> >>>> Are you running (R)STP ?
> >>>
> >>> I'm running RSTP. It doesn't look like an STP loop thing, since if I
> >>> isolate the offending machine from the network the flood stops. If it
> >>> were a topology/loop problem, it wouldn't stop unless the loop would
> >>> be physically broken.
> >>
> >> Correct. However it looks like some form of gratuitous ARP, amplified
> >> through a broadcast loop. Could it be a buggy network stack that
> >> triggers some interaction between the client and the AP that
> >> re-broadcasts it?
> >
> > Uhm, AFAIK gratuitous ARPs are essentially ARP replies, while here we're
> > seeing ARP requests.
> >
> > However, I certainly can't exclude a buggy OS, especially since we're
> > talking Apple here (see below).
> 
> 
> No - GARP is a request for own IP [1]. The idea is to invite response
> from some other host having the same IP, to avoid collisions.

I see now that both methods are in use, so here we're seeing gratuitous
_requests_ (which, contrary to my wrong understanding, seems indeed to be
the preferred choice). You are right, thanks for the clarification.

It seems both methods are in use, see eg

http://wiki.wireshark.org/Gratuitous_ARP
http://kb.juniper.net/InfoCenter/index?page=content&id=KB24349

or wikipedia:

"ARP may also be used as a simple announcement protocol. This is useful for
updating other hosts mapping of a hardware address when the senders IP
address or MAC address has changed. Such an announcement, also called a
gratuitous ARP message, is usually broadcast as an ARP request containing
the senders protocol address (SPA) in the target field (TPA=SPA), with the
target hardware address (THA) set to zero. An alternative is to broadcast
an ARP reply with the sender's hardware and protocol addresses (SHA and
SPA) duplicated in the target fields (TPA=SPA, THA=SHA).

An ARP announcement is not intended to solicit a reply; instead it updates
any cached entries in the ARP tables of other hosts that receive the
packet. The operation code may indicate a request or a reply because the
ARP standard specifies that the opcode is only processed after the ARP
table has been updated from the address fields."
 
> The GARP's are probably belonging to Apple Rendezvous (Zeroconf)
> protocol looking for an address to use, when the host has no other
> useful address. Are they for 169.254.x.y?

No, they are for the host's own IP address, but thanks for mentioning
zeroconf which gives further ideas for investigation.

[toc] | [prev] | [next] | [standalone]

#3209

pk wrote:
[....]
> 18:23:44.326912 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46

Is there ever any answer ?

(10.15.10.251 is at 3c:15:c2:c8:cb:11, or the likes ?)


-- 
Mostly harmless...
        -Douglas Adams

[toc] | [prev] | [next] | [standalone]

#3211

On Thu, 15 Jan 2015 10:12:08 +0100, Ralph Spitzner <rasp@spitzner.org>
wrote:

> pk wrote:
> [....]
> > 18:23:44.326912 3c:15:c2:c8:cb:11 > ff:ff:ff:ff:ff:ff, ethertype ARP
> > (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has
> > 10.15.10.251 (ff:ff:ff:ff:ff:ff) tell 10.15.10.251, length 46
> 
> Is there ever any answer ?
> 
> (10.15.10.251 is at 3c:15:c2:c8:cb:11, or the likes ?)
> 

I can't see any answer. The fact is, the machine with MAC 3c:15:c2:c8:cb:11
has *already* the address 10.15.10.251.

 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.networking

csiph-web