A TCP/UDP Load Balancer for 2 ISP connections
The problem:
Utilization of little-used Internet gateway in conjunction
with main line to improve network performance.
The hardware:
Old PC, 3 NICs, two connected to the ISP gateways and one
to the local LAN.
The goal:
One Internet connection is twice as fast as the other one.
Therefore we need to split outgoing connections in a 2:1 ratio so that,
for example, we can split downloads "fairly" among the two connections.
The solution:
Linux already offers everything that is needed to tackle
this in the form of the amazingly configurable iptables, netfilter and
iproute2.
This was implemented using Debian GNU/Linux, but any modern Linux
distribution with a recent kernel will do just fine.
The nitty gritty:
We use the connection tracking feature to mark/track
packets in a round-robin fashion so we can route them through the
desired NICs and achieve outgoing data load balancing.
Obviously this can be done with more than 2 ISPs or with different speed
ratios than 2:1.
netfilter/iptables project homepage: http://netfilter.org/
iproute2 project homepage: http://linuxfoundation.org/collaborate/workgroups/networking/iproute2/
1 #!/bin/bash 2 # 3 # TCP/UDP Load Balancer 4 # 5 # Made possible by the collective wisdom of the Internets 6 # Compiled by Jimmy Angelakos <vyruss@hellug.gr> 7 # 8 # eth0 is the LAN connection. 9 # eth1 is connected to ISP#1 with IP1 via gateway GW1 on network NET1. 10 # eth2 is connected to ISP#2 with IP2 via gateway GW2 on network NET2. 11 # DNS1 is DNS server for ISP#1 and DNS2 is for ISP#2. 12 13 14 # Your configuration variables: 15 IF1=eth0 16 IF2=eth1 17 IF3=eth2 18 IP1=192.168.22.99 19 IP2=192.168.99.99 20 GW1=192.168.22.2 21 GW2=192.168.99.1 22 NET1=192.168.22.0/24 23 NET2=192.168.99.0/24 24 DNS1=193.92.150.3 25 DNS2=195.170.0.1 26 27 # Create 2 new routing tables 28 if ! cat /etc/iproute2/rt_tables | grep -q '^251' 29 then 30 echo '251 T1' >> /etc/iproute2/rt_tables 31 fi 32 if ! cat /etc/iproute2/rt_tables | grep -q '^252' 33 then 34 echo '252 T2' >> /etc/iproute2/rt_tables 35 fi 36 ip route flush table T1 37 ip route flush table T2 38 39 # Associate routing tables with the different ISPs. 40 ip route add $NET1 dev $IF1 src $IP1 table T1 41 ip route add default via $GW1 table T1 42 ip route add $NET2 dev $IF2 src $IP2 table T2 43 ip route add default via $GW2 table T2 44 ip route add $NET1 dev $IF1 src $IP1 45 ip route add $NET2 dev $IF2 src $IP2 46 47 # Add your other networks to both tables. 48 ip route add 10.0.0.0/8 via 192.168.22.4 table T1 49 ip route add 10.0.0.0/8 via 192.168.22.4 table T2 50 51 # Delete old rules. 52 ip rule del from all fwmark 0x1 lookup T1 2>/dev/null 53 ip rule del from all fwmark 0x2 lookup T2 2>/dev/null 54 ip rule del from all fwmark 0x2 2>/dev/null 55 ip rule del from all fwmark 0x1 2>/dev/null 56 ip rule del from $IP1 57 ip rule del from $IP2 58 59 # Add new rules for connection tracking to our routing tables. 60 ip rule add from $IP1 table T1 61 ip rule add from $IP2 table T2 62 ip rule add fwmark 1 table T1 63 ip rule add fwmark 2 table T2 64 65 # You need these system settings. 66 echo 1 > /proc/sys/net/ipv4/ip_forward 67 for f in /proc/sys/net/ipv4/conf/*/rp_filter ; do echo 1 > $f ; done 68 69 # Clear iptables/netfilter settings. 70 iptables -F 71 iptables -X 72 iptables -t nat -F 73 iptables -t nat -X 74 iptables -t filter -F 75 iptables -t filter -X 76 iptables -t mangle -F 77 iptables -t mangle -X 78 79 # Set up connection tracking and Source NAT. 80 81 # Create two chains that will mark the packet. 82 iptables -t mangle -N MARK1 83 iptables -t mangle -A MARK1 -j MARK --set-mark 1 84 iptables -t mangle -A MARK1 -j CONNMARK --save-mark 85 iptables -t mangle -N MARK2 86 iptables -t mangle -A MARK2 -j MARK --set-mark 2 87 iptables -t mangle -A MARK2 -j CONNMARK --save-mark 88 89 iptables -t nat -N OUT_IF1 90 iptables -t nat -A OUT_IF1 -j SNAT --to-source $IP1 91 iptables -t nat -N OUT_IF2 92 iptables -t nat -A OUT_IF2 -j SNAT --to-source $IP2 93 94 iptables -t nat -A POSTROUTING -o $IF1 -j OUT_IF1 95 iptables -t nat -A POSTROUTING -o $IF2 -j OUT_IF2 96 97 iptables -t nat -N IN_IF1 98 iptables -t nat -A IN_IF1 -j MARK --set-mark 1 99 iptables -t nat -A IN_IF1 -j CONNMARK --save-mark 100 iptables -t nat -N IN_IF2 101 iptables -t nat -A IN_IF2 -j MARK --set-mark 2 102 iptables -t nat -A IN_IF2 -j CONNMARK --save-mark 103 104 iptables -t nat -A PREROUTING -i $IF1 -j IN_IF1 105 iptables -t nat -A PREROUTING -i $IF2 -j IN_IF2 106 107 # TCP balancing. ISP#1 has twice the connection speed of ISP#2, so in every 3 packets two should go to ISP#1 and one to ISP#2. 108 iptables -t mangle -A PREROUTING -i $IF3 -p tcp -m tcp -m state --state NEW -m statistic --mode nth --every 3 --packet 0 -j MARK1 109 iptables -t mangle -A PREROUTING -i $IF3 -p tcp -m tcp -m state --state NEW -m statistic --mode nth --every 3 --packet 1 -j MARK2 110 iptables -t mangle -A PREROUTING -i $IF3 -p tcp -m tcp -m state --state NEW -m statistic --mode nth --every 3 --packet 2 -j MARK1 111 112 # Restore mark on packets belonging to existing connections. 113 iptables -t mangle -A PREROUTING -i $IF3 -p tcp -m tcp -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark 114 115 # UDP balancing. Same story as above. 116 iptables -t mangle -A PREROUTING -i $IF3 -p udp -m udp -m statistic --mode nth --every 3 --packet 0 -j MARK1 117 iptables -t mangle -A PREROUTING -i $IF3 -p udp -m udp -m statistic --mode nth --every 3 --packet 1 -j MARK2 118 iptables -t mangle -A PREROUTING -i $IF3 -p udp -m udp -m statistic --mode nth --every 3 --packet 2 -j MARK1 119 120 # Send DNS requests through the correct ISP or they will fail. 121 iptables -t mangle -A PREROUTING -p tcp --dport 53 -d $DNS1 -j MARK1 122 iptables -t mangle -A PREROUTING -p udp --dport 53 -d $DNS1 -j MARK1 123 iptables -t mangle -A PREROUTING -p tcp --dport 53 -d $DNS2 -j MARK2 124 iptables -t mangle -A PREROUTING -p udp --dport 53 -d $DNS2 -j MARK2 125 126 ip route flush cache 127 128 # You're done! Now you can watch the load balancing taking place with: 129 # watch -n1 iptables -t mangle -nvL 130 # 131 # Enjoy!