Networking vendors have long offered a functionality for aggregating
bandwidth across multiple physical links to a switch. This allows a machine
(frequently a server) to treat multiple physical connections to switch units as
a single logical link. The standard moniker for this technology is IEEE
802.3ad, although it is known by the common names of trunking, port trunking
and link aggregation. The conventional use of bonding under linux is an
implementation of this link aggregation
A separate use of the same driver allows the kernel to present a single
logical interface for two physical links to two separate switches. Only one
link is used at any given time. By using media independent interface signal
failure to detect when a switch or link becomes unusable, the kernel can,
transparently to userspace and application layer services, fail to the backup
physical connection. Though not common, the failure of switches, network
interfaces, and cables can cause outages. As a component of high availability
planning, these bonding techniques can
help reduce the number of single points of failure.
For more information on bonding, see the Documentation/networking/bonding.txt from
the linux source code tree.
Link Aggregation
Bonding for link aggregation must be supported by
both endpoints. Two linux machines connected via crossover cables can take
advantage of link aggregation. A single machine connected with two physical
cables to a switch which supports port trunking can use link aggregation to the
switch. Any conventional switch will become ineffably confused by a hardware
address appearing on multiple ports simultaneously.
Example - Link aggregation
bonding
|
[root@real-server
root]# modprobe bonding
[root@real-server
root]# ip addr add 192.168.100.33/24 brd + dev bond0
[root@real-server
root]# ip link set dev bond0 up
[root@real-server
root]# ifenslave bond0 eth2 eth3
master has no hw
address assigned; getting one from slave!
The interface eth2
is up, shutting it down it to enslave it.
The interface eth3
is up, shutting it down it to enslave it.
[root@real-server
root]# ifenslave bond0 eth2 eth3
[root@real-server
root]# cat /proc/net/bond0/info
Bonding Mode: load
balancing (round-robin)
MII Status: up
MII Polling
Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface:
eth2
MII Status: up
Link Failure Count:
0
Slave Interface:
eth3
MII Status: up
Link Failure Count:
0
|
FIXME; Need an experiment here....maybe a tcpdump
to show how the management frames appear on the wire.
This Beowulf software page describes in a bit more detail the
rationale and a practical application of linux channel bonding (for link
aggregation).
2.5.2. High
Availability
Bonding support under linux is part of a high
availability solution. For an entry point into the complexity of high
availability in conjunction with linux, see the linux-ha.org site.
To guard against layer two (switch) and layer one (cable) failure, a machine
can be configured with multiple physical connections to separate switch devices
while presenting a single logical interface to userspace.
The name of the interface can be specified by the
user. It is commonly bond0 or something similar. As a logical
interface, it can be used in routing tables and by tcpdump.
The bond interface, when created, has no link layer
address. In the example below, an address is manually added to the interface.
See Example 2.12, “Link aggregation bonding” for an
example of the bonding driver reporting setting the link layer address when the
first device is enslaved to the bond (doesn't that sound cruel!).
Example - High availability
bonding
|
[root@real-server
root]# modprobe bonding mode=1 miimon=100 downdelay=200 updelay=200
[root@real-server
root]# ip link set dev bond0 addr 00:80:c8:e7:ab:5c
[root@real-server
root]# ip addr add 192.168.100.33/24 brd + dev bond0
[root@real-server
root]# ip link set dev bond0 up
[root@real-server
root]# ifenslave bond0 eth2 eth3
The interface eth2
is up, shutting it down it to enslave it.
The interface eth3
is up, shutting it down it to enslave it.
[root@real-server
root]# ip link show eth2 ; ip link show eth3 ; ip link show bond0
4: eth2:
<BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0
qlen 100
link/ether 00:80:c8:e7:ab:5c brd
ff:ff:ff:ff:ff:ff
5: eth3:
<BROADCAST,MULTICAST,NOARP,SLAVE,DEBUG,AUTOMEDIA,PORTSEL,NOTRAILERS,UP>
mtu 1500 qdisc pfifo_fast master bond0 qlen 100
link/ether 00:80:c8:e7:ab:5c brd
ff:ff:ff:ff:ff:ff
58: bond0:
<BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc noqueue
link/ether 00:80:c8:e7:ab:5c brd
ff:ff:ff:ff:ff:ff
|
Immediately noticeable, there is a new flag in the ip
link show output. The MASTER and SLAVE flags
clearly report the nature of the relationship between the interfaces. Also, the
Ethernet interfaces indicate the master interface via the keywords master
bond0.
Note also, that all three of the interfaces share
the same link layer address, 00:80:c8:e7:ab:5c.