Our experimental testbed consists of IBM ThinkPad laptops ng BSD/OS) and Pentium-based personal computers running BSD/OS 2.1 from bsdl. The machines are interconnected a10 TCP Rece Mbps ethernet and 915 MHz at&T WaveLANs [271,a 2 Mbps WaveLAN Y ning Silos) shared-medium wireless Lan with a raw signalling band (lossy link Pentium-based Pc width of 2 Mbps. The network topology for our experiments is shown in Figure 2. The peak throughput for TCP bulk transfers is 1. 5 Mbps in the local area testbed and 1 Mbps in the wide area testbed in the absence of congestion tional 16 Internet hops between the souree and base sta- or wireless losses. These testbed topologies represent typi Figure 2. Experimental topology. There were an addi- cal scenarios of wireless links and mobile hosts such as cel- tion during the WAN experiments lular wireless networks. In addition, our experiments focus on data transfer to the mobile host. which is the common We added TCP awareness to both the LL and LL-SMArT case for mobile applications (e.g, Web accesses) protocols, resulting in the LL-TCP-AWARE and lL- SMART-TCP-AWARE schemes. The LL-TCP-AWARE In order to measure the performance of the protocols under protocol is identical to the snoop protocol, while the LL- controlled conditions, we generate errors on the lossy link SMART-TCP-AWARE protocol uses SMART-based tech- using an exponentially distributed bit-error model. The niques for further optimization using selective repeat. LL- receiving entity on the lossy link generates an exponential SMART-TCP-AWARE iS the best link-layer protocol in our distribution for each bit-error rate and changes the tcp experiments- it performs local retransmissions based on checksum of the packet if the error generator determines selective acknowledgments and shields the sender from that the packet should be dropped. Losses are generated in duplicate acknowledgments caused by wireless losses both directions of the wireless channel. so tcp acknowl edgments are dropped too. The TCP data packet size in 3.3 Split-Connection Schemes experiments is 1400 bytes. We first measure and analyze the Like i-tcp our sPlit scheme uses an intermediate host to performance of the various protocols at an average error rate of one every 64 KBytes(this corresponds to a bit-error rate divide a TCP connection into two separate TCP connec- of about 1.9x10-6) Note that since the exponential distribu ions. The implementation avoids data copying in the inter tion has a standard deviation equal to its mean, there are mediate host by passing the pointers to the same buffer several occasions when multiple packets are lost in close between the two TCP connections. A variant of the SPLIT succession. We then report the results of some burst error approach we investigated, SPLIT-SMART, uses a SMART- situations, where between two and six packets are dropped based selective acknowledgment scheme on the wireless connection to perform selective retransmissions. There is formance of many of these protocols across a range of erro little chance of reordering of packets over the wireless con- rates from one every 16 KB to one every 256 KB. The nection since the intermediate host is only one hop away choice of the exponentially distributed error model is moti from the final destination vated by our desire to understand the precise dynamics of each protocol in response to a wireless loss, and is not an 4. Experimental Results attempt to empirically model a wireless channel. While the actual performance numbers will be a function of the exact In this section, we describe the experiments we performed error model, the relative performance is dependent on how and the results we obtained, including detailed explanations the protocol behaves after one or more losses in a single for observed performance. We start by describing the exper- TCP window. Thus, we expect our overall conclusions to be imental testbed and methodology. We then describe the per- pplicable under other patterns of wireless loss as well formance of the various link-layer, end-to-end and split- Finally, we believe that though wireless errors are generated artificially in our experiments, the use of a real testbed still valuable in that it introduces realistic effects such as 4.1 Experimental Methodolog wireless bandwidth limitation. media access contention We performed several experiments to determine the protocol processing delays, etc, which are hard to model mance and efficiency of each of the protocols. The realistically in a simulation cols were implemented as a set of modifications to the In our experiments, we attempt to ensure that losses are only OS TCP/IP(Reno)network stack. To ensure a fair basis for due to wireless errors(and not congestion). This allows us comparison, none of the protocols implementations intro- to focus on the effectiveness of the mechanisms in handling duce any additional data copying at intermediate points such losses. The WAN experiments are performed across 16 from sender to receiver
We added TCP awareness to both the LL and LL-SMART protocols, resulting in the LL-TCP-AWARE and LLSMART-TCP-AWARE schemes. The LL-TCP-AWARE protocol is identical to the snoop protocol, while the LLSMART-TCP-AWARE protocol uses SMART-based techniques for further optimization using selective repeat. LLSMART-TCP-AWARE is the best link-layer protocol in our experiments — it performs local retransmissions based on selective acknowledgments and shields the sender from duplicate acknowledgments caused by wireless losses. 3.3 Split-Connection Schemes Like I-TCP, our SPLIT scheme uses an intermediate host to divide a TCP connection into two separate TCP connections. The implementation avoids data copying in the intermediate host by passing the pointers to the same buffer between the two TCP connections. A variant of the SPLIT approach we investigated, SPLIT-SMART, uses a SMARTbased selective acknowledgment scheme on the wireless connection to perform selective retransmissions. There is little chance of reordering of packets over the wireless connection since the intermediate host is only one hop away from the final destination. 4. Experimental Results In this section, we describe the experiments we performed and the results we obtained, including detailed explanations for observed performance. We start by describing the experimental testbed and methodology. We then describe the performance of the various link-layer, end-to-end and splitconnection schemes. 4.1 Experimental Methodology We performed several experiments to determine the performance and efficiency of each of the protocols. The protocols were implemented as a set of modifications to the BSD/ OS TCP/IP (Reno) network stack. To ensure a fair basis for comparison, none of the protocols implementations introduce any additional data copying at intermediate points from sender to receiver. Our experimental testbed consists of IBM ThinkPad laptops and Pentium-based personal computers running BSD/OS 2.1 from BSDI. The machines are interconnected using a 10 Mbps Ethernet and 915 MHz AT&T WaveLANs [27], a shared-medium wireless LAN with a raw signalling bandwidth of 2 Mbps. The network topology for our experiments is shown in Figure 2. The peak throughput for TCP bulk transfers is 1.5 Mbps in the local area testbed and 1.35 Mbps in the wide area testbed in the absence of congestion or wireless losses. These testbed topologies represent typical scenarios of wireless links and mobile hosts, such as cellular wireless networks. In addition, our experiments focus on data transfer to the mobile host, which is the common case for mobile applications (e.g., Web accesses). In order to measure the performance of the protocols under controlled conditions, we generate errors on the lossy link using an exponentially distributed bit-error model. The receiving entity on the lossy link generates an exponential distribution for each bit-error rate and changes the TCP checksum of the packet if the error generator determines that the packet should be dropped. Losses are generated in both directions of the wireless channel, so TCP acknowledgments are dropped too. The TCP data packet size in our experiments is 1400 bytes. We first measure and analyze the performance of the various protocols at an average error rate of one every 64 KBytes (this corresponds to a bit-error rate of about 1.9x10-6 ). Note that since the exponential distribution has a standard deviation equal to its mean, there are several occasions when multiple packets are lost in close succession. We then report the results of some burst error situations, where between two and six packets are dropped in every burst (Section 4.5). Finally, we investigate the performance of many of these protocols across a range of error rates from one every 16 KB to one every 256 KB. The choice of the exponentially distributed error model is motivated by our desire to understand the precise dynamics of each protocol in response to a wireless loss, and is not an attempt to empirically model a wireless channel. While the actual performance numbers will be a function of the exact error model, the relative performance is dependent on how the protocol behaves after one or more losses in a single TCP window. Thus, we expect our overall conclusions to be applicable under other patterns of wireless loss as well. Finally, we believe that though wireless errors are generated artificially in our experiments, the use of a real testbed is still valuable in that it introduces realistic effects such as wireless bandwidth limitation, media access contention, protocol processing delays, etc., which are hard to model realistically in a simulation. In our experiments, we attempt to ensure that losses are only due to wireless errors (and not congestion). This allows us to focus on the effectiveness of the mechanisms in handling such losses. The WAN experiments are performed across 16 TCP Source 10 Mbps Ethernet TCP Receiver 2 Mbps WaveLAN (lossy link) Pentium-based PC running BSD/OS) Base Station (Pentium PC running BSD/OS) (Pentium laptop running BSD/OS) Figure 2. Experimental topology. There were an additional 16 Internet hops between the source and base station during the WAN experiments
Wired Goodput reless Goodput LAN: Absolute□ Percentage of max□ 100 WAN: Absolute Percentage of max Throughput 三 LL-TCP-AWARE LL-SMART LL-SMART-TCP-AWARE Figure 3. Performance of link-layer protocols: bit-error rate=1.9x10(I error/65536 bytes), socket buffer size= 32 KB. For each case there are two bars: the thick one corresponds to the scale on the left and denotes the throughput in Mbps: the thin one corresponds to the scale on the right and shows the throughput as a percentage of the maximum, i.e. in the absence of wireless errors(1.5 Mbps in the Lan environment and 1.35 Mbps in the WAN environment). Internet hops with minimal congestion" in order to study the the link and transport layers often lead to significant perfor impact of large delay-bandwidth products mance degradation. However, this is not the dominating effect when link layer schemes, such as LL, are used with Each run in the experiment consists of an 8 MByte transfer TCP Reno and its variants. These TCP implementations from the source to receiver across the wired net and the have coarse retransmission timeout granularities that WaveLAN link. We chose this rather long transfer size in order to limit the impact of transient behavior at the start of typically multiples of 500 ms, while link-layer protocols a TCP connection. During each run, we measure the typically have much finer timeout granularities. The real problem is that when packets are lost, link-layer protocol throughput at the receiver in Mbps, and the wired and wire- that do not attempt in-order delivery across the link(e.g less goodputs as percentages. In addition, all packet trans- LL) cause packets to reach the TCP receiver out-of-order missions on the ethernet and waveLan are recorded for This leads to the generation of duplicate acknowledgments nalysis using tcpdump [201, and the sender's TCP code by the TCP receiver, which causes the sender to invoke fast instrumented to record events such as coarse timeouts retransmission and recovery. This can potentially cause retransmission times, duplicate acknowledgment arrivals, degraded throughput and goodput, especially when the congestion window size changes, etc. The rest of this sec tion presents and discusses the results of these experiments delay-bandwidth product is large Our results substantiate this claim, as can be seen by com- 4.2 Link-Laver Protocols paring the LL and LL-TCP-AWARE results(Figure 3 and Traditional link-layer protocols operate independently of Table 2). For a packet size of 1400 bytes, a bit error rate of 1.9x10-0(1/65536 bytes)translates to a packet error rate of the higher-layer protocol, and consequently, do not neces- about 2.2 to 2.3%. Therefore, an optimal link-layer protocol sarily shield the sender from the lossy link. In spite of local that recovers from errors locally and does not compete with retransmissions, TCP performance could be poor for two TCP retransmissions should have a wireless goodput of tting of timers wo lay ers. an sary invocations of the TCP fast retransmission mechanism gestion. In the LAN experiments, the throughput difference between LL and LL-TCP-AWARE is about 10%. however due to out-of-order delivery of data In[10], the effects or the LL wireless goodput is only 95.5%, significantly less the first situation are simulated and analyzed for a TCP-like than LL- TCP-AWARE's wireless goodput of97.6%,which transport protocol (that closely tracks the round-trip time to set its retransmission timeout) and a reliable link-layer pro- is close to the maximum achievable goodput. When a loss tocol. The conclusion was that unless the packet loss rate is high(more than about 10%), competing retransmissions by atively quickly. However, enough packets are typically in transit to create more than 3 duplicate acknowledgments These duplicates eventually propagate to the sender and 2. WAN experiments across the US were performed between 10 trigger a fast retransmission and the associated congestion pm and 4 am, PST and we verified that no congestion losses control mechanisms. These fast retransmissions result in occurred in the runs reported
Internet hops with minimal congestion2 in order to study the impact of large delay-bandwidth products. Each run in the experiment consists of an 8 MByte transfer from the source to receiver across the wired net and the WaveLAN link. We chose this rather long transfer size in order to limit the impact of transient behavior at the start of a TCP connection. During each run, we measure the throughput at the receiver in Mbps, and the wired and wireless goodputs as percentages. In addition, all packet transmissions on the Ethernet and WaveLan are recorded for analysis using tcpdump [20], and the sender’s TCP code instrumented to record events such as coarse timeouts, retransmission times, duplicate acknowledgment arrivals, congestion window size changes, etc. The rest of this section presents and discusses the results of these experiments. 4.2 Link-Layer Protocols Traditional link-layer protocols operate independently of the higher-layer protocol, and consequently, do not necessarily shield the sender from the lossy link. In spite of local retransmissions, TCP performance could be poor for two reasons: (i) competing retransmissions caused by an incompatible setting of timers at the two layers, and (ii) unnecessary invocations of the TCP fast retransmission mechanism due to out-of-order delivery of data. In [10], the effects of the first situation are simulated and analyzed for a TCP-like transport protocol (that closely tracks the round-trip time to set its retransmission timeout) and a reliable link-layer protocol. The conclusion was that unless the packet loss rate is high (more than about 10%), competing retransmissions by the link and transport layers often lead to significant performance degradation. However, this is not the dominating effect when link layer schemes, such as LL, are used with TCP Reno and its variants. These TCP implementations have coarse retransmission timeout granularities that are typically multiples of 500 ms, while link-layer protocols typically have much finer timeout granularities. The real problem is that when packets are lost, link-layer protocols that do not attempt in-order delivery across the link (e.g., LL) cause packets to reach the TCP receiver out-of-order. This leads to the generation of duplicate acknowledgments by the TCP receiver, which causes the sender to invoke fast retransmission and recovery. This can potentially cause degraded throughput and goodput, especially when the delay-bandwidth product is large. Our results substantiate this claim, as can be seen by comparing the LL and LL-TCP-AWARE results (Figure 3 and Table 2). For a packet size of 1400 bytes, a bit error rate of 1.9x10-6 (1/65536 bytes) translates to a packet error rate of about 2.2 to 2.3%. Therefore, an optimal link-layer protocol that recovers from errors locally and does not compete with TCP retransmissions should have a wireless goodput of 97.7% and a wired goodput of 100% in the absence of congestion. In the LAN experiments, the throughput difference between LL and LL-TCP-AWARE is about 10%. However, the LL wireless goodput is only 95.5%, significantly less than LL-TCP-AWARE’s wireless goodput of 97.6%, which is close to the maximum achievable goodput. When a loss occurs, the LL protocol performs a local retransmission relatively quickly. However, enough packets are typically in transit to create more than 3 duplicate acknowledgments. These duplicates eventually propagate to the sender and trigger a fast retransmission and the associated congestion control mechanisms. These fast retransmissions result in 2. WAN experiments across the US were performed between 10 pm and 4 am, PST and we verified that no congestion losses occurred in the runs reported. LL LL-TCP-AWARE LL-SMART LL-SMART-TCP-AWARE Throughput (Mbps) Wireless Goodput LAN: Absolute Wired Goodput Figure 3. Performance of link-layer protocols: bit-error rate = 1.9x10-6 (1 error/65536 bytes), socket buffer size = 32 KB. For each case there are two bars: the thick one corresponds to the scale on the left and denotes the throughput in Mbps; the thin one corresponds to the scale on the right and shows the throughput as a percentage of the maximum, i.e. in the absence of wireless errors (1.5 Mbps in the LAN environment and 1.35 Mbps in the WAN environment). Throughput 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Percentage of max. WAN: Absolute Percentage of max. Throughput (% of maximum) 0 10 20 30 40 50 60 70 80 90 100 95.5 97.9 1.20 95.5 98.4 0.82 97.6 100.0 1.36 97.6 100.0 1.19 95.5 98.3 1.29 95.3 99.4 0.93 97.7 100.0 1.39 97.6 100.0 1.22