Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #200930 > unrolled thread

dual port cross over cable bonding

Back to article view | Back to linux.debian.user

Contents

  dual port cross over cable bonding Adam Weremczuk <adamw@matrixscience.com> - 2018-10-05 16:40 +0200
    Re: dual port cross over cable bonding Dan Ritter <dsr@randomstring.org> - 2018-10-05 16:50 +0200
      Re: dual port cross over cable bonding Dan Ritter <dsr@randomstring.org> - 2018-10-05 16:50 +0200
        [SOLVED] Re: dual port cross over cable bonding Adam Weremczuk <adamw@matrixscience.com> - 2018-10-05 17:20 +0200
    Re: dual port cross over cable bonding Dan Ritter <dsr@randomstring.org> - 2018-10-05 16:50 +0200
      Re: dual port cross over cable bonding Adam Weremczuk <adamw@matrixscience.com> - 2018-10-05 17:10 +0200
    Re: dual port cross over cable bonding Reco <recoverym4n@enotuniq.net> - 2018-10-05 17:20 +0200
    Re: dual port cross over cable bonding Dan Purgert <dan@djph.net> - 2018-10-05 17:20 +0200
      [SOLVED] Re: dual port cross over cable bonding Adam Weremczuk <adamw@matrixscience.com> - 2018-10-05 17:50 +0200

#200930 — dual port cross over cable bonding

Hello,

I have 2 servers running Proxmox 5.2 (based on Debian 9).

I've connected Ethernet ports between them with a pair of cross over cables.

When only 1 port on each and 1 cable are used connectivity looks fine 
with the following config:

-------------------------------------------------------

auto ens1f0
iface ens1f0 inet static
     address 192.168.200.1(2)
     netmask 255.255.255.252

-------------------------------------------------------

Then I connected 2 cables and attempted link aggregation:

-------------------------------------------------------

iface ens1f0 inet manual

iface ens1f1 inet manual

auto bond1
iface bond1 inet static
     slaves ens1f0 ens1f1
     address 192.168.200.1(2)
     netmask 255.255.255.252
     bond_miimon 100
#    bond_mode 802.3ad
     bond_mode balance-rr
     bond_xmit_hash_policy layer3+4

-------------------------------------------------------

Tried both 802.3ad and balance-rr modes.

AFAIK only these 2 provide link aggregation.

Ethtool appears to be happy:

ethtool bond1
Settings for bond1:
     Supported ports: [ ]
     Supported link modes:   Not reported
     Supported pause frame use: No
     Supports auto-negotiation: No
     Advertised link modes:  Not reported
     Advertised pause frame use: No
     Advertised auto-negotiation: No
     Speed: 2000Mb/s
     Duplex: Full
     Port: Other
     PHYAD: 0
     Transceiver: internal
     Auto-negotiation: off
     Link detected: yes

Unfortunately in either mode cross pinging fails with "Destination Host 
Unreachable".

Own interfaces ping ok.

The same configuration works fine against managed switch ports (LACP/LAG).

So my question is why this is not working and whether it's possible at all?

Regards,
Adam

[toc] | [next] | [standalone]

#200931

On Fri, Oct 05, 2018 at 03:45:14PM +0100, Adam Weremczuk wrote:
> Yes, 192.168.200.1 and 192.168.200.2 netmask 30
> All ports and cables work when tested 1-1 but not with my 2-2 bonding
> config.
> 
> > Looks like your LACP is correct but your IP addressing is wrong.
> > 
> > Do you have parentheses in it, or are you trying to suggest that
> > one is .1 and the other is .2?

Also, balance-rr is in conflict with layer3+4.

-dsr-

[toc] | [prev] | [next] | [standalone]

#200933

On Fri, Oct 05, 2018 at 10:46:15AM -0400, Dan Ritter wrote:
> On Fri, Oct 05, 2018 at 03:45:14PM +0100, Adam Weremczuk wrote:
> > Yes, 192.168.200.1 and 192.168.200.2 netmask 30
> > All ports and cables work when tested 1-1 but not with my 2-2 bonding
> > config.
> > 
> > > Looks like your LACP is correct but your IP addressing is wrong.
> > > 
> > > Do you have parentheses in it, or are you trying to suggest that
> > > one is .1 and the other is .2?
> 
> Also, balance-rr is in conflict with layer3+4.
> 

And I think slaves should be spelled "bond-slaves".

-dsr-

[toc] | [prev] | [next] | [standalone]

#200936 — [SOLVED] Re: dual port cross over cable bonding

My config was ok and it's working like a charm after both servers have 
been rebooted.
For some reason "systemctl restart networking" wasn't enough.

[toc] | [prev] | [next] | [standalone]

#200932

On Fri, Oct 05, 2018 at 03:14:15PM +0100, Adam Weremczuk wrote:
> Hello,
> 
> I have 2 servers running Proxmox 5.2 (based on Debian 9).
> 
> I've connected Ethernet ports between them with a pair of cross over cables.
> 
> When only 1 port on each and 1 cable are used connectivity looks fine with
> the following config:
> 
> -------------------------------------------------------
> 
> auto ens1f0
> iface ens1f0 inet static
>     address 192.168.200.1(2)
>     netmask 255.255.255.252
> 
> -------------------------------------------------------
> 
> Then I connected 2 cables and attempted link aggregation:
> 
> -------------------------------------------------------
> 
> iface ens1f0 inet manual
> 
> iface ens1f1 inet manual
> 
> auto bond1
> iface bond1 inet static
>     slaves ens1f0 ens1f1
>     address 192.168.200.1(2)
>     netmask 255.255.255.252
>     bond_miimon 100
> #    bond_mode 802.3ad
>     bond_mode balance-rr
>     bond_xmit_hash_policy layer3+4
> 
> -------------------------------------------------------
> 
> Tried both 802.3ad and balance-rr modes.
> 
> AFAIK only these 2 provide link aggregation.
> 
> Ethtool appears to be happy:
> 
> ethtool bond1
> Settings for bond1:
>     Supported ports: [ ]
>     Supported link modes:   Not reported
>     Supported pause frame use: No
>     Supports auto-negotiation: No
>     Advertised link modes:  Not reported
>     Advertised pause frame use: No
>     Advertised auto-negotiation: No
>     Speed: 2000Mb/s
>     Duplex: Full
>     Port: Other
>     PHYAD: 0
>     Transceiver: internal
>     Auto-negotiation: off
>     Link detected: yes
> 
> Unfortunately in either mode cross pinging fails with "Destination Host
> Unreachable".
> 
> Own interfaces ping ok.
> 
> The same configuration works fine against managed switch ports (LACP/LAG).
> 
> So my question is why this is not working and whether it's possible at all?

Looks like your LACP is correct but your IP addressing is wrong.

Do you have parentheses in it, or are you trying to suggest that
one is .1 and the other is .2?

-dsr-

[toc] | [prev] | [next] | [standalone]

#200934

Yes, 192.168.200.1 and 192.168.200.2 netmask 30
All ports and cables work when tested 1-1 but not with my 2-2 bonding 
config.

> Looks like your LACP is correct but your IP addressing is wrong.
>
> Do you have parentheses in it, or are you trying to suggest that
> one is .1 and the other is .2?
>
> -dsr-
>

[toc] | [prev] | [next] | [standalone]

#200937

	Hi.

On Fri, Oct 05, 2018 at 03:14:15PM +0100, Adam Weremczuk wrote:
> Hello,
> 
> I have 2 servers running Proxmox 5.2 (based on Debian 9).

Have you considered using OpenVSwitch for the task?


> I've connected Ethernet ports between them with a pair of cross over cables.

Ok.


> iface ens1f0 inet manual
> 
> iface ens1f1 inet manual

Not needed. ifupdown is smart enough to bring those up without these
lines.


> auto bond1
> iface bond1 inet static
>     slaves ens1f0 ens1f1

It's 'bond-slaves', not 'slaves'

>     address 192.168.200.1(2)
>     netmask 255.255.255.252
>     bond_miimon 100
> #    bond_mode 802.3ad
>     bond_mode balance-rr
>     bond_xmit_hash_policy layer3+4

Documentation/networking/bonding.txt.gz clearly states that:

xmit_hash_policy

    Selects the transmit hash policy to use for slave selection in
    balance-xor, 802.3ad, and tlb modes.

balance-rr and xmit_hash_policy do not play together.


> Tried both 802.3ad and balance-rr modes.

And predictably 802.3ad did not work for you.
You're using cross-cable = you lack LACP Partner.
No LACP probes = 802.3ad fails.


> AFAIK only these 2 provide link aggregation.

No. There is 'balance-tlb' that does it too, but it's useless in
cross-over setup.
Add any cheap unmanaged switch to the picture, and it will be different.


> Unfortunately in either mode cross pinging fails with "Destination Host Unreachable".

Leaving aside '802.3ad' that will never work for you, let's focus on
'balance-rr'.

Starting with something simple, have you confirmed via tcpdump that both
links receive ICMP from the other host?


> The same configuration works fine against managed switch ports (LACP/LAG).

Adding a hint here, OpenVSwitch can be LACP Partner. Linux kernel can be
Actor only.


> So my question is why this is not working and whether it's possible at all?

802.3ad - both hosts are unable to negotiate links for ingress traffic.
balance-rr - my best guess so far goes into ARP filtering direction.

Reco

[toc] | [prev] | [next] | [standalone]

#200938

Adam Weremczuk wrote:
> Hello,
>
> I have 2 servers running Proxmox 5.2 (based on Debian 9).
>
> I've connected Ethernet ports between them with a pair of cross over
> cables.

Are these "just" 10/100 cards?  I mean, if they're gbit capable,
crossover cables violate the spec, and you're limiting them to 100m
only...

-- 
|_|O|_| Registered Linux user #585947
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5  4AEE 8E11 DDF3 1279 A281

[toc] | [prev] | [next] | [standalone]

#200940 — [SOLVED] Re: dual port cross over cable bonding

A single dual port gigabit card on each server.

I'm more concerned about performance than redundancy.

The bond operates in 2 gigabit mode:

ethtool bond1
(...)
     Speed: 2000Mb/s
     Duplex: Full
(...)

My working config (will play with it a bit more):

auto bond1
iface bond1 inet static
     slaves ens1f0 ens1f1
     address 192.168.200.1
     netmask 255.255.255.252
     bond_miimon 100
     bond_mode balance-rr
     bond_xmit_hash_policy layer3+4

I've done a simple some performance test - copied a file over ssh (which 
adds own overhead).

90-100 MB/s over a single link and 150-160MB/s over a dual bond

So it's definitely working and making a substantial difference.

Unplugging one cable doesn't break the transfer; speed temporarily 
decreases but then goes up again after cable is reinserted.


On 05/10/18 16:01, Dan Purgert wrote:
> Are these "just" 10/100 cards?  I mean, if they're gbit capable,
> crossover cables violate the spec, and you're limiting them to 100m
> only...
>

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web