Techno Blog Just another Technology Blog

15Apr/114

NIC Bonding

Bonding two physical Ethernet interfaces together to make one logical interface is a great way to provide redundancy or increase bandwidth. In this example I demonstrate how to configure bonding in mode 1 which is the (active-backup) configuration.

Modes

Mode Description
0 (balance-rr) Sets a round-robin policy for fault tolerance and load balancing. Transmissions are received and sent out sequentially on each bonded slave interface beginning with the first one available.
1 (active-backup) Sets an active-backup policy for fault tolerance. Transmissions are received and sent out via the first available bonded slave interface. Another bonded slave interface is only used if the active bonded slave interface fails.
2 (balance-xor) Sets an XOR (exclusive-or) policy for fault tolerance and load balancing. Using this method, the interface matches up the incoming request's MAC address with the MAC address for one of the slave NICs. Once this link is established, transmissions are sent out sequentially beginning with the first available interface.
3 (broadcast) Sets a broadcast policy for fault tolerance. All transmissions are sent on all slave interfaces.
4 (802.3ad) Sets an IEEE 802.3ad dynamic link aggregation policy. Creates aggregation groups that share the same speed and duplex settings. Transmits and receives on all slaves in the active aggregator. Requires a switch that is 802.3ad compliant.
5 (balance-tlb) Sets a Transmit Load Balancing (TLB) policy for fault tolerance and load balancing. The outgoing traffic is distributed according to the current load on each slave interface. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed slave.
6 (balance-alb) Sets an Active Load Balancing (ALB) policy for fault tolerance and load balancing. Includes transmit and receive load balancing for IPV4 traffic. Receive load balancing is achieved through ARP negotiation.

Mode information obtained from RHEL 5 Deployment Guide.

  1. Stop Networking
  2. $ /etc/init.d/network stop
    

  3. Create bond0 Configuration
  4. $ vi /etc/sysconfig/network-scripts/ifcfg-bond0
    
    DEVICE=bond0
    IPADDR=192.168.1.20
    NETWORK=192.168.1.0
    NETMASK=255.255.255.0
    USERCTL=no
    BOOTPROTO=none
    ONBOOT=yes
    BONDING_OPTS="mode=1 miimon=100"
    

  5. Create eth0 Configuration
  6. $ vi /etc/sysconfig/network-scripts/ifcfg-eth0
    
    DEVICE=eth0
    HWADDR=08:00:27:7F:80:9A
    BOOTPROTO=none
    USERCTL=no
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    

  7. Create eth1 Configuration
  8. $ vi /etc/sysconfig/network-scripts/ifcfg-eth1
    
    DEVICE=eth1
    HWADDR=08:00:27:d1:44:ab
    BOOTPROTO=none
    USERCTL=no
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    

  9. Add bond0 to modprobe.conf
  10. $ echo "alias bond0 bonding" >> /etc/modprobe.conf
    

  11. Load bonding module
  12. $ modprobe -v bonding
    

  13. Verify bonding modules is loaded
  14. $ lsmod | grep bond
    bonding               139865  0
    ipv6                  435617  24 bonding,ip6t_REJECT
    

  15. Start Up Networking
  16. $ /etc/init.d/network start
    Bringing up loopback interface:                            [  OK  ]
    Bringing up interface bond0:                               [  OK  ]
    

  17. Display Network Interfaces
  18. $ ifconfig bond0
    bond0     Link encap:Ethernet  HWaddr 08:00:27:7F:80:9A
              inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: fe80::a00:27ff:fe7f:809a/64 Scope:Link
              UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
              RX packets:848 errors:0 dropped:0 overruns:0 frame:0
              TX packets:484 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:101787 (99.4 KiB)  TX bytes:68666 (67.0 KiB)
    

    $ ifconfig eth0
    eth0      Link encap:Ethernet  HWaddr 08:00:27:7F:80:9A
              UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
              RX packets:573 errors:0 dropped:0 overruns:0 frame:0
              TX packets:504 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:69166 (67.5 KiB)  TX bytes:71358 (69.6 KiB)
    

    $ ifconfig eth1
    eth1      Link encap:Ethernet  HWaddr 08:00:27:7F:80:9A
              UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
              RX packets:300 errors:0 dropped:0 overruns:0 frame:0
              TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:34801 (33.9 KiB)  TX bytes:192 (192.0 b)
    

  19. View Status of bond0 master and eth0/eth1 slaves
  20. $ cat /proc/net/bonding/bond0
    Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)
    
    Bonding Mode: fault-tolerance (active-backup)
    Primary Slave: None
    Currently Active Slave: eth0
    MII Status: up
    MII Polling Interval (ms): 100
    Up Delay (ms): 0
    Down Delay (ms): 0
    
    Slave Interface: eth0
    MII Status: up
    Link Failure Count: 0
    Permanent HW addr: 08:00:27:7f:80:9a
    
    Slave Interface: eth1
    MII Status: up
    Link Failure Count: 0
    Permanent HW addr: 08:00:27:d1:44:ab
    

Failover Testing

  1. View slaves
  2. $ cat /sys/class/net/bond0/bonding/slaves
    eth0 eth1
    

  3. Determine Active Slave
  4. $ cat /sys/class/net/bond0/bonding/active_slave
    eth0
    

  5. Remove active slave from bond0
  6. $ echo -eth0 > /sys/class/net/bond0/bonding/slaves
    

  7. View slaves and see that eth0 is no longer present
  8. $ cat /sys/class/net/bond0/bonding/slaves
    eth1
    
    $ cat /proc/net/bonding/bond0
    Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)
    
    Bonding Mode: fault-tolerance (active-backup)
    Primary Slave: None
    Currently Active Slave: eth1
    MII Status: up
    MII Polling Interval (ms): 100
    Up Delay (ms): 0
    Down Delay (ms): 0
    
    Slave Interface: eth1
    MII Status: up
    Link Failure Count: 0
    Permanent HW addr: 08:00:27:d1:44:ab
    

  9. Add eth0 back to bond0
  10. $ echo +eth0 > /sys/class/net/bond0/bonding/slaves
    

  11. Verify both slaves are present again
  12. $ cat /sys/class/net/bond0/bonding/slaves
    eth0 eth1
    
    $ cat /proc/net/bonding/bond0
    Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)
    
    Bonding Mode: fault-tolerance (active-backup)
    Primary Slave: None
    Currently Active Slave: eth1
    MII Status: up
    MII Polling Interval (ms): 100
    Up Delay (ms): 0
    Down Delay (ms): 0
    
    Slave Interface: eth0
    MII Status: up
    Link Failure Count: 0
    Permanent HW addr: 08:00:27:7f:80:9a
    
    Slave Interface: eth1
    MII Status: up
    Link Failure Count: 0
    Permanent HW addr: 08:00:27:d1:44:ab
    

    Keep an eye on "MII Status" after you add it back. It might take a few seconds to switch from "down" to "up".

Scripting Health Checks
A few files which provide some good information if you writing a scripted health check for your master or slave interfaces.

  • Get Link Status of bond0
  • $ cat /sys/class/net/bond0/bonding/mii_status
    up

  • Check which slave is active
  • $ cat /sys/class/net/bond0/bonding/active_slave
    eth0

  • Check for transmit or receive errors on one of the physical interfaces
  • $ cat /sys/class/net/bond0/slave_eth0/statistics/tx_errors
    0
    $ cat /sys/class/net/bond0/slave_eth0/statistics/rx_errors
    0
Comments (4) Trackbacks (0)
  1. when I am trying to add the slave interface with the command

    echo +eth2 > /sys/class/net/bond0/bonding/slaves

    throwing error as ” -bash: echo: write error: Operation not permitted”

    • I am not positive why your write to /sys/class/net/bond0/bonding/slaves is not working, however this is really an old school sort of method anyway and I didn’t figure anyone besides be would be reading this. 🙂 If I am going to be helping someone else I would suggest the “ifenslave” command instead. It is easier and safer than writing to /sys.

      Some example usage for you:

      # Show information about all interfaces
      ifenslave -a

      # Change active slave to eth1
      ifenslave -c bond0 eth1

      # Remove slave interface eth2 from the bonding device
      ifenslave -d bond0 eth2

      # Add slave interface eth2 back to bond0
      ifenslave bond0 eth2

      See “ifenslave –help” for more information or of course it’s man page. In Red Hat, CentOS, Fedora the “ifenslave” command is provided by the “iputils” package.

      Sorry for the late reply. Hope this helps someone.

  2. Awesome document

  3. Thanks for useful topic.

    I wonder there is any way to change the bonding interface to broadcast mode after it has been up.


Leave a comment

No trackbacks yet.