Mersault/MultiLink PPP

From FBSD_tips

Multilink PPP is a technology that lets you bond multiple PPP sessions into one logical bundle that can then function as a single, larger, connection.

1 Concept
2 What You Need
3 Userland PPP Configuration
4 MPD Configuration
- 4.1 mpd4
- 4.2 mpd5
5 PF Configuration
6 Split Packets vs. Round Robin
7 Further Discussion
8 Linux Configuration

[edit] Concept

Multilink PPP should not be confused with load balancing (though it often is). Load balancing happens at OSI layer 3, the IP layer. With load balancing each TCP stream is sent down one of the links, and thus each single TCP stream can only move as quickly as the link it's been sent down. While your aggregate speed can be equal to the speed of all of your links combined, no one TCP stream is going to go faster than the link it's using.

Multilink PPP happens entirely in OSI layer 2, the data link layer. You establish multiple PPP sessions between yourself and a peer (ISP) and bond them together into a bundle. This bundle is presented to the IP layer as one logical connection. The bundle can be configured to either send consecutive packets down each link in a round robin fashion, or it can actually split each packet in half and send a portion up each link, to be reconstituted at the peer end of the connection.

Multilink PPP was defined in RFC 1990, which provides an extension to PPP that allows multiple connections to be bonded together. It is commonly done with T1's, where the 23 B channels of the T1 can be bonded together for 1.5 megabits of bandwidth.

The TCP/IP Guide is an excellent resource for all things TCP/IP (and more, since PPP is data link layer). The section on Multilink PPP is very useful for understanding the concepts.

[edit] What You Need

First thing you are going to need is a computer running FreeBSD. Instructions for that are far beyond the scope of this document, and frankly there's plenty of excellent documentation out there already, namely the FreeBSD Handbook. You can use the various online resources for setting up your FreeBSD machine as a router, following the instructions as if you were setting up a regular PPPoE connection. Then just modify your ppp or mpd config file for MLPPP.

Typically you're going to want to have (at least) two phone lines with DSL service on both of them. If each of your DSL connections has a sync rate of 6 megabits down, 800 kilobits up, then the bonded connection will have 12 megabits down, and 1.6 megabits up (this is before network overhead is considered).

Strictly speaking, you can do MLPPP with only one DSL connection, but this wont double your bandwidth. It can however let you use an MTU and MRU greater than 1492, if your ISP supports split packets over MLPPP, which can be useful in some circumstances.

Speaking of the ISP, you need an ISP that supports MLPPP on their routers. This is rather uncommon, so you'll want to be sure to check with the ISP before you go ahead and order your DSL lines. Expect to have to speak to several levels in tech support, and expect tech support to have to speak to the engineers for a real answer. In Canada TekSavvy will support MLPPP, but be sure to call and verify that you're in a supported area.

For hardware, you'll want a WAN NIC for each DSL connection (in addition to any LAN or DMZ NIC's in your router), as well as a modem for each DSL connection. It may be possible to use PCI ADSL cards, but I have no experience with them.

[edit] Userland PPP Configuration

Userland PPP is the default version of PPP included in FreeBSD. It's simple and effective, and it gets the job done. All the processing happens in userland, which does incur a small performance hit in terms of CPU usage, but this should be negligible on any computer manufactured in this millennium.

Here's a simple ppp.conf that should work for most people:

 defaut:
   set log Phase Chat LCP IPCP CCP tun command
 ispname:
   set authname AuthName
   set authkey AuthPasswd
   set device PPPoE:xl0 PPPoE:xl1
   set dial
   set login
   set logout
   set hangup
   set speed sync
   enable tcpmssfixup
   enable mssfixup
   disable lqr ipv6cp acfcomp pred1 protocomp vjcomp deflate chap81 pap
   deny lqr acfcomp pred1 protocomp vjcomp deflate mppe chap
   accept pap chap
   set mtu max 1440
   set ifaddr 10.1.1.2/0 10.1.1.1/0 0 0
   add! default hisaddr
   set timeout 0
   set mru max 1440
   set mrru 1528
   set server /var/run/ppp-tun%d 0177
   clone 1,2
   link deflink remove
   link 1,2 set mode ddial
   link 1 set device PPPoE:xl0
   link 2 set device PPPoE:xl1

This configuration was posted by an MLPPP user to the DSL Reports TekSavvy forums:

ppp.conf:

default:  
  set log phase tun command  
  disable ipv6
ispname:  
  set authname AuthName  
  set authkey AuthPasswd  
  set dial  
  set login  
  set logout  
  set hangup  
  set speed sync  
  set server /var/run/ppp-tun%d 0177  
  set ifaddr X.X.X.X Y.Y.Y.Y 255.255.255.255  
  add! default hisaddr  
  set timeout 0  
  disable lqr ipv6cp acfcomp pred1 protocomp vjcomp deflate ipv6  
  deny lqr acfcomp pred1 protocomp vjcomp deflate mppe  
  accept pap chap  
  set mtu max 1492  
  set mru max 1486  
  # For CISCO use 1524 mrru and for Juniper use 1590 mrru  
  set mrru 1590  
  clone 1,2  
  link deflink remove  
  link 1,2 set mode ddial  
  link 1 set device PPPoE:xl0  
  link 2 set device PPPoE:xl1

ppp.linkup:

MYADDR:  
!bg /sbin/ifconfig tun0 mtu 1486

The most important line in these configurations is the 'set mrru' line. This is the line that actually tells ppp to create a multilink connection. As for the proper values for 'set mtu', 'set mru', and 'set mrru', that is going to depend on your ISP and the connection you have to them. You'll also need to modify the MSS of outbound packets to account for the different MTU and MRU settings. pf has builtin support for MSS mangling, while tcpmssd can be used with ipfw to achieve the same result.

[edit] MPD Configuration

Mpd is the Multilink PPP Daemon. It has been designed specifically for MLPPP, and so should have some additional features some users may find useful. The author has only used it mostly to prove that it works, but has not had any need to use the advanced features. Mpd supports better control of the individual links than userland ppp.

These mpd configurations are for a 3 link setup. These are known to work with the CVS versions of mpd4 and mpd5 as of the end of January 2008. Any release of mpd4 or mpd5 since then should include fixes that were a result of some testing I did with the developer.

[edit] mpd4

mpd.conf:

startup:
  set console ip 127.0.0.1
  set console user admin password
  set console open
default:
  new sam l0 l1
  set bundle enable multilink
  set bundle enable round-robin
  set iface route default

mpd.links:

l0:
  set link type pppoe
  set pppoe iface xl0
  set pppoe disable incoming
  set pppoe enable originate
  set auth authname AuthName 
  set auth password AuthPasswd
  set link max-redial 0
  open
l1:
  set link type pppoe
  set pppoe iface xl1
  set pppoe disable incoming
  set pppoe enable originate
  set auth authname AuthName 
  set auth password AuthPasswd
  set link max-redial 0
  open
l2:
  set link type pppoe
  set pppoe iface fxp0
  set pppoe disable incoming
  set pppoe enable originate
  set auth authname AuthName 
  set auth password AuthPasswd
  set link max-redial 0
  open

[edit] mpd5

mpd.conf:

default:
  create bundle static B1
  set iface route default
  create link static L1 pppoe
  set pppoe iface xl0
  set auth authname AuthName
  set auth password AuthPasswd
  set link enable multilink
  set link max-redial 0
  set link action bundle B1
  open

  create link static L2 pppoe
  set pppoe iface xl1
  set auth authname AuthName
  set auth password AuthPasswd
  set link enable multilink
  set link max-redial 0
  set link action bundle B1
  open

  create link static L3 pppoe
  set pppoe iface fxp0
  set auth authname AuthName
  set auth password AuthPasswd
  set link enable multilink
  set link max-redial 0
  set link action bundle B1
  open

[edit] PF Configuration

These three lines help with MLPPP packet reassembly

scrub in on $ext_if all fragment reassemble min-ttl 15 max-mss 1440
scrub in on $ext_if all no-df
scrub on $ext_if all reassemble tcp

[edit] Split Packets vs. Round Robin

MLPPP can work in two ways. The simplest is when packets are sent whole down each link in a round robin fashion. This will double your bandwidth but wont allow you to bypass restrictions on the packet size (1492 for PPPoE).

The other way is to split the packet in two, and send each half down each link. This is the preferred way to work, as the connections are truly bonded. Along each link the packet has a further 6 byte header added to it, but because the packet has been split in half the packet as a whole can be greater than 1492. Packet splitting should result in less jitter than the round robin setup, though this will be most noticeable on the upload side of the connection.