Essentially they've got four ADSL links:
- eth0: 111.111.111.111 (main uncapped line)
- eth1: 100.100.100.11
- eth2: 100.100.100.22
- eth3: 100.100.100.33
eth0 is the default route where all traffic goes through, and eth3 is used for something else.
They're using Squid as their proxy server, and want all traffic for a specific Squid ACL (let's call it "employees") to be load balanced between eth1 and eth2.
My biggest worry was in how to set this up without having to alter the default route on the system, but - as is always the case with Linux - there is a way if you look hard enough.
My first thought was to use iptables to mark packets, then set up an iproute2 rule to pick up on those packets and forward them to the relevant routing table. Unfortunately, the best one could do with this method is manipulate all http traffic, however, as mentioned, only the traffic for specific users must be load balanced.
Obviously, this method won't work.
The key to the solution is Squid's TCP_OUTGOING_TOS configuration directive. This directive allows one to set the TOS value in outgoing IP packets on a per-ACL basis. What this means is that if you have an ACL called "employees", you can have all traffic generated by the users in that ACL have the TOS set to an arbitrary value. You can then use an iproute2 rule to pick up on all packets with that value set and do whatever you want with them. Bingo!
Firstly, we need to create three routing tables: ADSL1, ADSL2 and BALANCE. This is usually done in /etc/iproute2/rt_tables.
Then we begin adding routing information:
# Assumes 100.100.100.10 is the address of the router on eth1
ip route add 100.100.100.9/29 dev eth1 src 100.100.100.11 table ADSL1
ip route add default via 100.100.100.10 table ADSL1
# Assumes 100.100.100.21 is the address of the router on eth2
ip route add 100.100.100.20/29 dev eth2 src 100.100.100.22 table ADSL2
ip route add default via 100.100.100.21 table ADSL2
Now we add some rules
# eth1 traffic goes to table ADSL1
ip rule add from 100.100.100.11 table ADSL1
# eth2 traffic goes to table ADSL2
ip rule add from 100.100.100.22 table ADSL2
# Squid sets desired traffic TOS. Marked traffic goes to table BALANCE
ip rule add tos 0x0c table BALANCE
And finally we create the multilink route in the table BALANCE.
ip route add default scope global table BALANCE nexthop via 100.100.100.10 dev eth1 weight 1 nexthop via 100.100.100.21 dev eth2 weight 1
Then finally, in your Squid configuration, simply add the configuration directive
tcp_outgoing_tos 0x0c employees
That's pretty much it. iptraf confirms that all http traffic for the desired ACL gets balanced over eth1 and eth2. Obviously the load balancing won't be perfect due to route caching, etc. But it probably is the most beautiful solution possible without splashing out on a pricey Cisco router or some such.
For more information, be sure to read the Linux Advanced Routing & Traffic Control HOWTO.
