Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #188744 > unrolled thread

Ethernet card locking up when acting as virtual bridge

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Ethernet card locking up when acting as virtual bridge Andrew Wood <andrewjameswood@ymail.com> - 2017-11-08 12:00 +0100
    Re: Ethernet card locking up when acting as virtual bridge Christian Seiler <christian@iwakd.de> - 2017-11-08 16:00 +0100
      Re: Ethernet card locking up when acting as virtual bridge Andrew W <andrewjameswood@ymail.com> - 2017-11-08 20:40 +0100
    Re: Ethernet card locking up when acting as virtual bridge David Christensen <dpchrist@holgerdanske.com> - 2017-11-09 00:30 +0100
      Re: Ethernet card locking up when acting as virtual bridge Nicholas Geovanis <nickgeovanis@gmail.com> - 2017-11-09 18:40 +0100
      Re: Ethernet card locking up when acting as virtual bridge Dan Ritter <dsr@randomstring.org> - 2017-11-09 19:00 +0100
        Re: Ethernet card locking up when acting as virtual bridge David Christensen <dpchrist@holgerdanske.com> - 2017-11-09 20:20 +0100
        Re: Ethernet card locking up when acting as virtual bridge Michael Stone <mstone@debian.org> - 2017-11-09 21:00 +0100
          Re: Ethernet card locking up when acting as virtual bridge Andrew W <andrewjameswood@ymail.com> - 2017-11-10 19:30 +0100
            Re: Ethernet card locking up when acting as virtual bridge Christian Seiler <christian@iwakd.de> - 2017-11-10 19:30 +0100

#188744 — Ethernet card locking up when acting as virtual bridge

[Multipart message — attachments visible in raw view] — view raw

Im trying to use a 3Com Etherlink Model 3C905C to provide network access for some virtual machines running under QEMU.
The machine has a Realtek Gigabit Ethernet controller on the motherboard which I use solely for the hosts network interface. I've added a 3Com PCI card to act as the interface solely for the VMs allowing them to have IPs on the real network rather than using QEMUs NAT.
My configuration is below. Initially it worked fine, except that once in a while the card would seemingly 'lock up' i.e no VMs could get network access but unplugging and replugging the Cat 5 cable fixed it.
Recently however the issue has been occuring more and more, sometimes some VMs can get network access and not others. Ive tried swapping the card for an identical model but its made no difference. Im not sure if it something to do with my config (maybe the made up MAC addresses?) or if its a bug in the cards driver or firmware.
Config is below eth0 is the hosts Realtek port, enp3s0 is the 3Com used for the VMs
The host OS is StretchAm I doing something wrong or is this a bug?
Many thanks
======================/etc/network/interfaces:
# The loopback network interfaceauto loiface lo inet loopback



auto eth0iface eth0 inet static        address 192.168.253.201        netmask 255.255.255.0        broadcast 192.168.253.255        dns-nameservers 192.168.253.254        up /sbin/route add default gw 192.168.253.254 dev eth0


auto enp3s0auto br1iface br1 inet dhcp        bridge_ports enp3s0        bridge_stp off        bridge_fd 0        bridge_maxwait 0======================

The script I use to start QEMU contains lines like this for each VM:
#host debian9test vnc on port 5902, give it virtual hub port br1p1tunctl -u root -t br1p1 #create TAP device br1p1brctl addif br1 br1p1 #add br1p1 as a port on bridge br1kvm -hda /var/qemuvm/debian9test.img  -m 1024 -usb \-net nic,macaddr=52:54:00:00:00:02 -net tap,ifname=br1p1 -vnc 192.168.253.201:2 &

#host bitsafe vnc on port 5904, give it virtual hub port br1p2tunctl -u root -t br1p2  #create TAP device br1p2brctl addif br1 br1p2  # add br1p2 as a port on bridge br1kvm -hda /dev/disk/by-id/ata-TOSHIBA_HDWJ110_37K1P2AET \-hdb /dev/disk/by-id/ata-TOSHIBA_HDWJ110_37UNTL5IT \-m 1024 -usb -net nic,macaddr=52:54:00:00:00:04  \-net tap,ifname=br1p2 -vnc 192.168.253.201:4 &

#host gatekeeper (RADIUS) vnc on port 5905, give it virtual hub port br1p3tunctl -u root -t br1p3 #create TAP device br1p3brctl addif br1 br1p3 #add br1p3 as a port on bridge br1kvm -hda /var/qemuvm/debian-gatekeeper.img \-m 1024 -usb -net nic,macaddr=52:54:00:00:00:05 \-net tap,ifname=br1p3 -vnc 192.168.253.201:5 &

======================Output from ifconfig:
br1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.253.80  netmask 255.255.255.0  broadcast 192.168.253.255        inet6 fe80::250:daff:fe3b:9a4d  prefixlen 64  scopeid 0x20<link>        ether 00:50:da:3b:9a:4d  txqueuelen 1000  (Ethernet)        RX packets 22633  bytes 1224397 (1.1 MiB)        RX errors 0  dropped 9  overruns 0  frame 0        TX packets 159  bytes 18064 (17.6 KiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
br1p1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        ether 76:73:87:89:3c:f1  txqueuelen 1000  (Ethernet)        RX packets 1391  bytes 97413 (95.1 KiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 22130  bytes 1514391 (1.4 MiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
br1p2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        ether 82:e0:2d:0d:d8:26  txqueuelen 1000  (Ethernet)        RX packets 4416  bytes 337769 (329.8 KiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 25436  bytes 1728395 (1.6 MiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
br1p3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        ether 26:67:8d:04:5c:b9  txqueuelen 1000  (Ethernet)        RX packets 329  bytes 38930 (38.0 KiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 22583  bytes 1553522 (1.4 MiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        ether 00:50:da:3b:9a:4d  txqueuelen 1000  (Ethernet)        RX packets 27632  bytes 2188088 (2.0 MiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 6303  bytes 493295 (481.7 KiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0        device interrupt 20  base 0x5000
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500        inet 192.168.253.201  netmask 255.255.255.0  broadcast 192.168.253.255        inet6 fe80::21f:d0ff:fe08:8bc6  prefixlen 64  scopeid 0x20<link>        ether 00:1f:d0:08:8b:c6  txqueuelen 1000  (Ethernet)        RX packets 22949  bytes 20198348 (19.2 MiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 24046  bytes 1880515 (1.7 MiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        inet6 ::1  prefixlen 128  scopeid 0x10<host>        loop  txqueuelen 1  (Local Loopback)        RX packets 26302  bytes 1783961 (1.7 MiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 26302  bytes 1783961 (1.7 MiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[toc] | [next] | [standalone]

#188750

Am 2017-11-08 11:54, schrieb Andrew Wood:
> My configuration is below. Initially it worked fine, except that once
> in a while the card would seemingly 'lock up' i.e no VMs could get
> network access but unplugging and replugging the Cat 5 cable fixed it.
> 
> Recently however the issue has been occuring more and more, sometimes
> some VMs can get network access and not others. Ive tried swapping the
> card for an identical model but its made no difference. Im not sure if
> it something to do with my config (maybe the made up MAC addresses?)
> or if its a bug in the cards driver or firmware.
> 
> Config is below eth0 is the hosts Realtek port, enp3s0 is the 3Com
> used for the VMs
> 
> iface br1 inet dhcp

No idea if this is the problem or not, but you are using DHCP for the
bridge interface. Maybe the DHCP client's management of the bridge
interface interferes with this? (DHCP clients like to 'take over' an
interface and may set flags that don't work properly.) I don't think
I've ever used a DHCP client on a bridge before other than for short
testing purposes.

Is is possible for you to try a static IP on this interface and see
if that solves your problem?

If that doesn't help:

  - The Linux bridge interface will always use the numerically lowest
    (in a Big-Endian sense) MAC address that's configured as the MAC
    address for the bridge. If you're doing virtualization, you
    should stick to MAC addresses that are very high for this reason,
    so that the network card's own address always wins out. Many
    virtualization tools ensure that the MAC addresses they generate
    start with 0xfe for this reason. (This might also confuse DHCP
    clients btw. if the MAC address of the interface changes behind
    their back.)

  - When the problem occurs: do the VMs at least see each other? If
    that doesn't even work, your problem isn't your network card,
    but something else.

  - Anything in the kernel log?
    If not, is there maybe a debug option for your specific network
    driver you could enable so that something is added to the logs?

  - Try running 'ip l' and 'ip a' both when the problem occurs and
    when it doesn't occur and see if there's any difference in the
    output of either of these.

  - You said that unplugging and replugging the cable made it work
    again - maybe the link sensing between your network card and the
    switch / whatever you have on the other side is broken for some
    reason? Could you try to force the right speed / duplex settings
    via ethtool and see if that helps?

  - Maybe your network card supports various offloading features
    (such as TCP checksums) but when used as a switch they don't
    always work properly - you could try disabling them (also via
    ethtool, see the manpage).

  - You said that you've tried replacing the network card - have you
    tried replacing the cable? I've had some weird experiences with
    broken network cables. (For example, very strange intermittent
    packet loss in some instances, which cause very weird effects in
    higher-level protocols.)

Regards,
Christian

[toc] | [prev] | [next] | [standalone]

#188767

On 08/11/2017 14:59, Christian Seiler wrote:
>
> Is is possible for you to try a static IP on this interface and see
> if that solves your problem?
>
Ive cleared the dhcp on br1 (and not assigned a static, left it with no 
IP) and so far its working OK
I will leave it a couple of days and see if it has indeed fixed the problem
Thanks Christian for your comprehensive reply and help.

Regards
Andrew

[toc] | [prev] | [next] | [standalone]

#188791

On 11/08/17 02:54, Andrew Wood wrote:
> 3Com Etherlink Model 3C905C

That card is *old* -- it brings back memories. :-)  And, 3Com is gone. 
Is there any FOSS support for 3Com stuff?


Intel supports FOSS on their products, which means their products are 
much more likely to work correctly on Debian GNU/Linux (and others). 
So, I buy Intel stuff.


David

[toc] | [prev] | [next] | [standalone]

#188815

[Multipart message — attachments visible in raw view] — view raw

On Wed, Nov 8, 2017 at 5:21 PM, David Christensen <dpchrist@holgerdanske.com
> wrote:

> On 11/08/17 02:54, Andrew Wood wrote:
>
>> 3Com Etherlink Model 3C905C
>>
>
> That card is *old* -- it brings back memories. :-)  And, 3Com is gone. Is
> there any FOSS support for 3Com stuff?


You know you're getting old when they put photos of your old network gear
on webpages
as if they were museum pieces, eg. https://en.wikipedia.org/wiki/3Com

David
>
>

[toc] | [prev] | [next] | [standalone]

#188817

On Wed, Nov 08, 2017 at 03:21:00PM -0800, David Christensen wrote:
> On 11/08/17 02:54, Andrew Wood wrote:
> > 3Com Etherlink Model 3C905C
> 
> That card is *old* -- it brings back memories. :-)  And, 3Com is gone. Is
> there any FOSS support for 3Com stuff?

This is one of the classic fast-ethernet cards, supported by the original
Beowulf cluster implementation:

http://webhome.phy.duke.edu/~rgb/brahma/Resources/beowulf/linux/drivers/vortex.html

One would expect that any bugs left in the kernel driver
(net/ethernet/3com/3c59x.ko) would be hammered flat by now.

kernel.org says it's had four changes in the last year.

-dsr-

[toc] | [prev] | [next] | [standalone]

#188824

On 11/09/17 09:55, Dan Ritter wrote:
> On Wed, Nov 08, 2017 at 03:21:00PM -0800, David Christensen wrote:
>> On 11/08/17 02:54, Andrew Wood wrote:
>>> 3Com Etherlink Model 3C905C
>>
>> That card is *old* -- it brings back memories. :-)  And, 3Com is gone. Is
>> there any FOSS support for 3Com stuff?
> 
> This is one of the classic fast-ethernet cards, supported by the original
> Beowulf cluster implementation:
> 
> http://webhome.phy.duke.edu/~rgb/brahma/Resources/beowulf/linux/drivers/vortex.html
> 
> One would expect that any bugs left in the kernel driver
> (net/ethernet/3com/3c59x.ko) would be hammered flat by now.

Kernel changes can and do break device drivers -- I wrote a driver for 
Linux 2.4.18 back in the day; it broke on 2.4.19.


The grim reality is that everything has to be re-validated on every 
kernel release.


> kernel.org says it's had four changes in the last year.

Good -- the OP's hardware *is* supported on Linux.  :-)


So, the next place to look is configuration settings.  It looks like 
Christian Seiler provided some good advice.


David

[toc] | [prev] | [next] | [standalone]

#188828

On Thu, Nov 09, 2017 at 12:55:05PM -0500, Dan Ritter wrote:
>On Wed, Nov 08, 2017 at 03:21:00PM -0800, David Christensen wrote:
>> On 11/08/17 02:54, Andrew Wood wrote:
>> > 3Com Etherlink Model 3C905C
>>
>> That card is *old* -- it brings back memories. :-)  And, 3Com is gone. Is
>> there any FOSS support for 3Com stuff?
>
>This is one of the classic fast-ethernet cards, supported by the original
>Beowulf cluster implementation:
>
>http://webhome.phy.duke.edu/~rgb/brahma/Resources/beowulf/linux/drivers/vortex.html
>
>One would expect that any bugs left in the kernel driver
>(net/ethernet/3com/3c59x.ko) would be hammered flat by now.

I'd wonder if it's reached the point tha that the hardware is failing.  
At any rate, a generic realtek gigabit card (free after rebate!) will 
run rings around what was a good card in the late 90s but functionally 
obsolete today.

Back to the original config: did you know you could add the built-in 
adapter to the bridge, give your IP address to the bridge rather than 
directly to the built-in interface, and use the VMs the same way you are 
now without a second NIC?

Mike Stone

[toc] | [prev] | [next] | [standalone]

#188853

On 09/11/2017 19:52, Michael Stone wrote:
> I'd wonder if it's reached the point tha that the hardware is failing.
I was a bit premature in thinking it was fixed!  Tried 3 different 3Com 
cards now plus a Linksys with a Via chipset but got identical behavour 
on all.

> Back to the original config: did you know you could add the built-in 
> adapter to the bridge, give your IP address to the bridge rather than 
> directly to the built-in interface, and use the VMs the same way you 
> are now without a second NIC?
>
>
Yes but as I was experiementing I wanted to keep it separate so if I 
screwed up I could at least still SSH into the host.

The weird thing is I have 3 VMs, all started with the same commands just 
the made up MAC address and virtual bridge port numbers differing. One 
of them has never given a problem it always works, the other two seem to 
alternate, one "goes offline" the other comes online, then vice versa, 
yet if I use VNC to bring up the console for the VM it is still running 
ok and I cant see any log messages on it indicating that the network has 
gone down.

The arping command  is indicating that when a VM goes offline it is not 
responding to ARP requests over the network yet it does from another VM 
and can be connected to perfectly from another VM

I then had a brainwave, I had a similar issue with a Cisco wifi access 
point where some wifi clients could connect ok, others would connect but 
then there was no traffic flowing.

That turned out to be the Cisco switch on the other end which is an 
ESW500 series Small Business Switch (i.e web gui only no IOS CLI). On 
there there is a 'Smartports Wizard' which allows you to set a 'role' 
for each port and unless it is set to 'Access Point' I got the strange 
behaviour.

  As the virtual bridge and wifi access point are pretty much the same 
thing in principle I tried changing the port role to 'Access Point' as 
well and it seems to have solved the issue. Ive no idea what that is 
changing in the switch config, and when I asked on the Cisco forum I 
couldnt get an answer but it seems to be the cause of the trouble.

[toc] | [prev] | [next] | [standalone]

#188854

Hi there,

On 11/10/2017 07:24 PM, Andrew W wrote:
> That turned out to be the Cisco switch on the other end which is an
> ESW500 series Small Business Switch (i.e web gui only no IOS CLI). On
> there there is a 'Smartports Wizard' which allows you to set a 'role'
> for each port and unless it is set to 'Access Point' I got the
> strange behaviour.
> 
> As the virtual bridge and wifi access point are pretty much the same
> thing in principle I tried changing the port role to 'Access Point'
> as well and it seems to have solved the issue. Ive no idea what that
> is changing in the switch config, and when I asked on the Cisco forum
> I couldnt get an answer but it seems to be the cause of the trouble.

I presume that if a port is not set in "Access point' mode the switch
will assume that packets coming from that port will always come from
the same MAC address, and since the VMs you have all have differing
addresses this causes the switch to silently drop packets from a
different MAC address. And depending on how they actually implemented
that in detail the behavior may be quite weird.

Regards,
Christian

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web