The Design Philosophy of the DARPA Internet Protocols David d. clark Massachusetts Institute of Technology Laboratory for Computer Science Cambridge, MA 02139 (Originally published in Proc. SIGCOMM 88, Computer Communication Review Vol. 18, No 4 August1988,pp.106-114) abstract architecture into the IP and TCP layers. This seems basic to the design, but was also not a part of th The Internet protocol suite, TCP/IP, was first proposed fifteen years ago. It was developed by the Defense original proposal. These changes in the Internet design Advanced Research Projects Agency(DARPA), and rose through the repeated pattern of implementation has been used widely in military and commercial and testing that occurred before the standards were set systems. While there have been papers and The Internet architecture is still evolving. Sometimes a specifications that describe how the protocols work, it is new extension challenges one of the design principles sometimes difficult to deduce from these why the but in any case an understanding of the history of the protocol is as it is. For example, the Internet protocol is design provides a necessary context for current design based on a connectionless or datagram mode of service extensions. The connectionless configuration of Iso The motivation for this has been greatly misunderstood protocols has also been colored by the history of the This to capture some of the early Internet suite. so an understanding of the Internet design reasoning which shaped the Internet protocols philosophy may be helpful to those working with Iso 1. Introduction This paper catalogs one view of the original objectives of the Internet architecture. and discusses the relation For the last 15 years, the Advanced Research Projects between these goals and the important features of the Agency of the U.s. Department of Defense has been developing a suite of protocols for packet switched networking. These protocols, which include the Internet 2. Fundamental gor Protocol(IP), and the Transmission Control Protocol (TCP), are now U.S. Department of Defense standards The top level goal for the DARPA Internet Architecture for internetworking, and are in wide use in the was to develop an effective technique for multiplexed commercial networking environment. The ideas utilization of existing interconnected networks. Some developed in this effort have also influenced other elaboration is appropriate to make clear the meaning of protocol suites, most importantly the connectionless that goal. configuration of the Iso protocols The components of the Internet were networks, which While specific information on the dod protocols is were to be interconnected to provide some larger fairly generally available,,, it is sometimes difficult to service. The original goal was to connect together the determine the motivation and reasoning which led to the riginal ARPANET with the ArPa packet radio network, in order to give users on the packet radio network access to the large service machines on the In fact, the design philosophy has evolved considerably ARPANET At the time it was assumed that there would from the first proposal to the current standards. For be other sorts of networks to interconnect, although the example, the idea of the datagram, or connectionless local area network had not yet emerged service, does not receive particular emphasis in the first paper, but has come to be the defining characteristic of An alternative to interconnecting existing networks would have been to design a unified system which This work was supported in part by the defense Advanced research Projects agesorpsrRted) anteF eth fae differen transmission media, a ACM SIGCOMM Computer Communication Review
ACM SIGCOMM -1- Computer Communication Review The Design Philosophy of the DARPA Internet Protocols David D. Clark* Massachusetts Institute of Technology Laboratory for Computer Science Cambridge, MA. 02139 (Originally published in Proc. SIGCOMM ‘88, Computer Communication Review Vol. 18, No. 4, August 1988, pp. 106–114) *This work was supported in part by the Defense Advanced Research Projects Agency (DARPA) under Contract No. N00014-83-K-0125 Abstract The Internet protocol suite, TCP/IP, was first proposed fifteen years ago. It was developed by the Defense Advanced Research Projects Agency (DARPA), and has been used widely in military and commercial systems. While there have been papers and specifications that describe how the protocols work, it is sometimes difficult to deduce from these why the protocol is as it is. For example, the Internet protocol is based on a connectionless or datagram mode of service. The motivation for this has been greatly misunderstood. This paper attempts to capture some of the early reasoning which shaped the Internet protocols. 1. Introduction For the last 15 years1 , the Advanced Research Projects Agency of the U.S. Department of Defense has been developing a suite of protocols for packet switched networking. These protocols, which include the Internet Protocol (IP), and the Transmission Control Protocol (TCP), are now U.S. Department of Defense standards for internetworking, and are in wide use in the commercial networking environment. The ideas developed in this effort have also influenced other protocol suites, most importantly the connectionless configuration of the ISO protocols2,3,4 . While specific information on the DOD protocols is fairly generally available5,6,7 , it is sometimes difficult to determine the motivation and reasoning which led to the design. In fact, the design philosophy has evolved considerably from the first proposal to the current standards. For example, the idea of the datagram, or connectionless service, does not receive particular emphasis in the first paper, but has come to be the defining characteristic of the protocol. Another example is the layering of the architecture into the IP and TCP layers. This seems basic to the design, but was also not a part of the original proposal. These changes in the Internet design arose through the repeated pattern of implementation and testing that occurred before the standards were set. The Internet architecture is still evolving. Sometimes a new extension challenges one of the design principles, but in any case an understanding of the history of the design provides a necessary context for current design extensions. The connectionless configuration of ISO protocols has also been colored by the history of the Internet suite, so an understanding of the Internet design philosophy may be helpful to those working with ISO. This paper catalogs one view of the original objectives of the Internet architecture, and discusses the relation between these goals and the important features of the protocols. 2. Fundamental Goal The top level goal for the DARPA Internet Architecture was to develop an effective technique for multiplexed utilization of existing interconnected networks. Some elaboration is appropriate to make clear the meaning of that goal. The components of the Internet were networks, which were to be interconnected to provide some larger service. The original goal was to connect together the original ARPANET8 with the ARPA packet radio network9,10, in order to give users on the packet radio network access to the large service machines on the ARPANET. At the time it was assumed that there would be other sorts of networks to interconnect, although the local area network had not yet emerged. An alternative to interconnecting existing networks would have been to design a unified system which incorporated a variety of different transmission media, a
multi-media network. while this might have permitted a 6. The Internet architecture must permit host higher degree of integration, and thus better attachment with a low level of effort performance, it was incorporate the then existing network architectures if 7. The resources used in the internet architecture must Internet was to be useful in a practical sense. Further, be accountable networks represent administrative boundaries of This set of goals might seem to be nothing more than a control, and it was an ambition of this project to come checklist of all the desirable network features. It is to grips with the problem of integrating a number of important to understand that these goals are in order of separately administrated entities into a common utility importance, and an entirely different The technique selected for multiplexing was packet architecture would result if the order were changed. For switching. An alternative such as circuit switching could example, since this network was designed to operate in have been considered, but the applications being a military context, which implied the possibility of a supported, such as remote login, were naturally served hostile environment, survivability was put as a first by the packet switching paradigm, and the networks goal, and accountability as a last goal. During wartime which were to be integrated together in this project were one is less concerned with detailed accounting of packet switching networks. So packet switching was resources used than with mustering whatever resources accepted as a fundamental component of the Internet are available and rapidly deploying them in an architecture operational manner. While the architects of the Internet were mindful of accountability, the problem received The final aspect of this fundamental goal was the very little attention during the early stages of the design, sumption of the particular technique for inter- and is only now being considered. An architecture connecting these networks. Since the technique of store primarily for commercial deployment would clearly nd forward packet switching, as demonstrated in the place these goals at the opposite end of the list previous DARPA project, the ARPANET, was well understood, the top level assumption was that networks Similarly, the goal that the architecture be cost effective would be interconnected by a layer of Internet pa is clearly on the list, but below certain other goals, such switches, which were called gateways as distributed management, or support of a wide variety of networks. Other protocol suites, including some of From these assumptions comes the more popular commercial architectures, have been structure of the Internet: a packet switched communica- optimized to a particular kind of network, for example a tions facility in which a number of distinguishable long haul store and forward network built of medium networks are connected together using packet communi speed telephone lines, and deliver a very cost effective cations processors called gateways which implement a solution in this context, in exchange for dealing store and forward packet forwarding algorithm somewhat poorly with other kinds of nets, such as local area nets 3. Second level goals The reader should consider carefully the above list of The top level goal stated in the previous section goals, and recognize that this is not a"motherhood"list ontains the word"effective, without offering any but a set of priorities which strongly colored the design definition of what an effective interconnection must decisions within the Internet architecture. The following achieve. The following list summarizes a more detailed sections discuss the relationship between this list and set of goals which were established for the Internet the features of the internet architecture 1. Internet communication must continue despite loss 4. Survivability in the Face of Failure of networks or gateways The most important goal on the list is that the Internet 2. The Internet must support multiple should continue to supply communications service, even communications service though networks and gateways are failing. In particular, this goal was interpreted to mean that if two entities are 3. The Internet architecture must accommodate a communicating over the Internet, and some failure of networks es the Internet to be temporarily disrupted and The Internet architecture must permit distributed reconfigured to reconstitute the service, then the entities management of its resources communicating should be able to continue without having to reestablish or reset the high level state of their 5. The Internet architecture must be cost effective conversation. More concretely, at the service interface of the transport layer, this architecture provides no ACM SIGCOMM Computer Communication Review
ACM SIGCOMM -2- Computer Communication Review multi-media network. While this might have permitted a higher degree of integration, and thus better performance, it was felt that it was necessary to incorporate the then existing network architectures if Internet was to be useful in a practical sense. Further, networks represent administrative boundaries of control, and it was an ambition of this project to come to grips with the problem of integrating a number of separately administrated entities into a common utility. The technique selected for multiplexing was packet switching. An alternative such as circuit switching could have been considered, but the applications being supported, such as remote login, were naturally served by the packet switching paradigm, and the networks which were to be integrated together in this project were packet switching networks. So packet switching was accepted as a fundamental component of the Internet architecture. The final aspect of this fundamental goal was the assumption of the particular technique for interconnecting these networks. Since the technique of store and forward packet switching, as demonstrated in the previous DARPA project, the ARPANET, was well understood, the top level assumption was that networks would be interconnected by a layer of Internet packet switches, which were called gateways. From these assumptions comes the fundamental structure of the Internet: a packet switched communications facility in which a number of distinguishable networks are connected together using packet communications processors called gateways which implement a store and forward packet forwarding algorithm. 3. Second Level Goals The top level goal stated in the previous section contains the word "effective," without offering any definition of what an effective interconnection must achieve. The following list summarizes a more detailed set of goals which were established for the Internet architecture. 1. Internet communication must continue despite loss of networks or gateways. 2. The Internet must support multiple types of communications service. 3. The Internet architecture must accommodate a variety of networks. 4. The Internet architecture must permit distributed management of its resources. 5. The Internet architecture must be cost effective. 6. The Internet architecture must permit host attachment with a low level of effort. 7. The resources used in the internet architecture must be accountable. This set of goals might seem to be nothing more than a checklist of all the desirable network features. It is important to understand that these goals are in order of importance, and an entirely different network architecture would result if the order were changed. For example, since this network was designed to operate in a military context, which implied the possibility of a hostile environment, survivability was put as a first goal, and accountability as a last goal. During wartime, one is less concerned with detailed accounting of resources used than with mustering whatever resources are available and rapidly deploying them in an operational manner. While the architects of the Internet were mindful of accountability, the problem received very little attention during the early stages of the design, and is only now being considered. An architecture primarily for commercial deployment would clearly place these goals at the opposite end of the list. Similarly, the goal that the architecture be cost effective is clearly on the list, but below certain other goals, such as distributed management, or support of a wide variety of networks. Other protocol suites, including some of the more popular commercial architectures, have been optimized to a particular kind of network, for example a long haul store and forward network built of medium speed telephone lines, and deliver a very cost effective solution in this context, in exchange for dealing somewhat poorly with other kinds of nets, such as local area nets. The reader should consider carefully the above list of goals, and recognize that this is not a "motherhood" list, but a set of priorities which strongly colored the design decisions within the Internet architecture. The following sections discuss the relationship between this list and the features of the Internet. 4. Survivability in the Face of Failure The most important goal on the list is that the Internet should continue to supply communications service, even though networks and gateways are failing. In particular, this goal was interpreted to mean that if two entities are communicating over the Internet, and some failure causes the Internet to be temporarily disrupted and reconfigured to reconstitute the service, then the entities communicating should be able to continue without having to reestablish or reset the high level state of their conversation. More concretely, at the service interface of the transport layer, this architecture provides no
facility to communicate to the client of the transport lgorithms that ensure the sequencing and service that the synchronization between the sender and acknowledgment of data fail, applications on that the receiver may have been lost. It was an assumption in this architecture that synchronization would never be lost unless there was no physical path over which ar Despite the fact that survivability is the first goal in the sort of communication could be achieved. In other list, it is still second to the top level goal of words, at the top of transport, there is only one failure, interconnection of existing networks. A more survivable and it is total partition. The architecture was to mask technology might have resulted from a single multi completely any transient failure media network design. For example, the Internet makes very weak assumptions To achieve this goal, the state information which report that it has failed. Internet is thus forced to detect describes the on-going conversation must be protected network failures using Internet level mechanisms, with Specific examples of state information would be the the potential for a slower and less specific error number of packets transmitted, the number of packets detect acknowledged, or the number of outstanding flow control permissions. If the lower layers of the archi- 5. Types of Service tecture lose this information, they will not be able to tell if data has been lost, and the application layer will have The second goal of the Internet architecture is that it to cope with the loss of synchrony. This architecture should support, at the transport service level, a variety insisted that this disruption not occur, which meant that of types of service. Different types of service are the state information must be protected from loss distinguished by differing requirements for such things as speed, latency and reliability. The traditional type of In some network architectures. this state is stored in the service is the bi-directional reliable delivery of data intermediate packet switching nodes of the network. In This service. which is sometimes called a "virtual this case, to protect the information from loss, it must circuit"service, is appropriate for such applications as replicated. Because of the distributed nature of the remote login or file transfer. It was the first service replication, algorithms to ensure robust replication are provided in the Internet architecture, using the themselves difficult to build. and few networks with Transmission Control Protocol (TCP).It was early distributed state information provide any sort of recognized that even this service had multiple variants protection against failure. The alternative, which this because remote login required a service with low delay architecture chose, is to take this information and gather in delivery, but low requirements for bandwidth, while it at the endpoint of the net, at the entity which is file transfer was less concerned with delay, but very utilizing the service of the network. I call this approach concerned with high throughput. TCP attempted to to reliability" fate-sharing. The fate-sharing model provide both these types of service. suggests that it is acceptable to lose the state information associated with an entity if, at the same The initial concept of TCP was that it could be general time, the entity itself is lost. Specifically, information enough to support any needed type of service. However, about transport level synchronization is stored in the as the full range of needed services became clear. it host which is attached to the net and using its seemed too difficult to build support for all of them into There are two important advantages to fate-sharing over The first example of a service outside the range of TCP replication. First, fate-sharing protects against any was support for XNET, the cross-Internet debugger number of intermediate failures, whereas replication can TCP did not seem a suitable transport for XNET for only protect against a certain number (less than the several reasons. First, a debugger protocol should not number of replicated copies). Second, fate-sharing is be reliable. This conclusion may seem odd, but unde conditions of stress or failure (which may be exactly hen a debugger is needed) asking for reliable There are two consequences to the fate-sharing communications may prevent any communications at approach to survivability. First, the intermediate packet all. It is much better to build a service which can deal switching nodes, or gateways, must not have any with whatever gets through, rather than insisting that essential state information about on-going connections every byte sent be delivered in order. Second, if TCP is Instead, they are stateless packet switches, a class of general enough to deal with a broad range of clients, it network design sometimes called a"datagram"network. is presumably somewhat complex. Again, it seemed Secondly, rather more trust is placed in the host wrong to expect support for this complexity in a machine than in an architecture where the network debugging environment, which may lack even basic ensures the reliable delivery of data. If the host resident ACM SIGCOMM Computer Communication Review
ACM SIGCOMM -3- Computer Communication Review facility to communicate to the client of the transport service that the synchronization between the sender and the receiver may have been lost. It was an assumption in this architecture that synchronization would never be lost unless there was no physical path over which any sort of communication could be achieved. In other words, at the top of transport, there is only one failure, and it is total partition. The architecture was to mask completely any transient failure. To achieve this goal, the state information which describes the on-going conversation must be protected. Specific examples of state information would be the number of packets transmitted, the number of packets acknowledged, or the number of outstanding flow control permissions. If the lower layers of the architecture lose this information, they will not be able to tell if data has been lost, and the application layer will have to cope with the loss of synchrony. This architecture insisted that this disruption not occur, which meant that the state information must be protected from loss. In some network architectures, this state is stored in the intermediate packet switching nodes of the network. In this case, to protect the information from loss, it must replicated. Because of the distributed nature of the replication, algorithms to ensure robust replication are themselves difficult to build, and few networks with distributed state information provide any sort of protection against failure. The alternative, which this architecture chose, is to take this information and gather it at the endpoint of the net, at the entity which is utilizing the service of the network. I call this approach to reliability "fate-sharing." The fate-sharing model suggests that it is acceptable to lose the state information associated with an entity if, at the same time, the entity itself is lost. Specifically, information about transport level synchronization is stored in the host which is attached to the net and using its communication service. There are two important advantages to fate-sharing over replication. First, fate-sharing protects against any number of intermediate failures, whereas replication can only protect against a certain number (less than the number of replicated copies). Second, fate-sharing is much easier to engineer than replication. There are two consequences to the fate-sharing approach to survivability. First, the intermediate packet switching nodes, or gateways, must not have any essential state information about on-going connections. Instead, they are stateless packet switches, a class of network design sometimes called a "datagram" network. Secondly, rather more trust is placed in the host machine than in an architecture where the network ensures the reliable delivery of data. If the host resident algorithms that ensure the sequencing and acknowledgment of data fail, applications on that machine are prevented from operation. Despite the fact that survivability is the first goal in the list, it is still second to the top level goal of interconnection of existing networks. A more survivable technology might have resulted from a single multimedia network design. For example, the Internet makes very weak assumptions about the ability of a network to report that it has failed. Internet is thus forced to detect network failures using Internet level mechanisms, with the potential for a slower and less specific error detection. 5. Types of Service The second goal of the Internet architecture is that it should support, at the transport service level, a variety of types of service. Different types of service are distinguished by differing requirements for such things as speed, latency and reliability. The traditional type of service is the bi-directional reliable delivery of data. This service, which is sometimes called a "virtual circuit" service, is appropriate for such applications as remote login or file transfer. It was the first service provided in the Internet architecture, using the Transmission Control Protocol (TCP)11. It was early recognized that even this service had multiple variants, because remote login required a service with low delay in delivery, but low requirements for bandwidth, while file transfer was less concerned with delay, but very concerned with high throughput. TCP attempted to provide both these types of service. The initial concept of TCP was that it could be general enough to support any needed type of service. However, as the full range of needed services became clear, it seemed too difficult to build support for all of them into one protocol. The first example of a service outside the range of TCP was support for XNET12, the cross-Internet debugger. TCP did not seem a suitable transport for XNET for several reasons. First, a debugger protocol should not be reliable. This conclusion may seem odd, but under conditions of stress or failure (which may be exactly when a debugger is needed) asking for reliable communications may prevent any communications at all. It is much better to build a service which can deal with whatever gets through, rather than insisting that every byte sent be delivered in order. Second, if TCP is general enough to deal with a broad range of clients, it is presumably somewhat complex. Again, it seemed wrong to expect support for this complexity in a debugging environment, which may lack even basic
services expected in an operating system(e.g. suppo Protocol (UDP) was created to provide a application- for timers. )So XNET was designed to run directly level interface to the basic datagram service of Internet p of the datagram service provided by Internet The architecture did not wish to assume that the Another service which did not fit tCP was real time underlying networks themselves support multiple types delivery of digitized speech, which was needed to of services, because this would violate the goal of using support the teleconferencing aspect of command and existing networks. Instead, the hope was that multiple control applications. In real time digital speech, the types of service could be constructed out of the basic primary requirement is not a reliable service, but a datagram building block using algorithms within the service which minimizes and smoothes the delay in the host and the gateway. For example, (although this is not delivery of packets. The application layer is digitizing done in most current implementations) it is possible to e an ng the resulting bits, and take datagrams which are associated with a controlled sending them out across the network on a regular basis delay but unreliable service and place them at the head They must arrive at the receiver at a regular basis in of the transmission queues unless their lifetime has order to be converted back to the analog signal. If expired, in which case they would be discarded; while packets do not arrive when expected, it is impossible to packets associated with reliable streams would be reassemble the signal in real time. A surprising placed at the back of the queues, but never discarded observation about the control of variation in delay is no matter how long they had been in the net that the most serious source of delay in networks is the mechanism to provide reliable delivery. a typical It proved more difficult than first hoped to provide reliable transport protocol responds to a missing packet multiple types of service without explicit support from by requesting a retransmission and delaying the delivery the underlying networks. The most serious problem was of any subsequent packets until the lost packet has been that networks designed with one particular type of retransmitted. It then delivers that packet and all service in mind were not flexible enough to support maining ones in sequence. The delay while this occurs other services. Most commonly, a network will have can be many times the round trip delivery time of the been designed under the assumption that it should net, and may completely disrupt the speech reassembly deliver reliable service, and will inject delays as a part algorithm. In contrast, it is very easy to cope with an of producing reliable service, whether or not this occasional missing packet. The missing speech can reliability is desired. The interface behavior defined by simply be replaced by a short period of silence, which X 25, for example, implies reliable delivery, and there in most cases does not impair the intelligibility of the is no way to turn this feature off. Therefore, although speech to the listening human. If it does, high level error Internet operates successfully over X 25 networks it correction can occur. and the listener can ask the cannot deliver the desired variability of type service in speaker to repeat the da that context. Other networks which have an intrinsic datagram service are much more flexible in the type of It was thus decided, fairly early in the development of service they will permit, but these networks are much the Internet architecture, that more than one transport less common, especially in the long-haul context. service would be required, and the architecture must be prepared to tolerate simultaneously transports which 6. Varieties of Networks wish to constrain reliability, delay, or bandwidth, at a It was very important for the success of the Internet architecture that it be able to incorporate and utilize a This goal caused TCP and IP, which originally had been wide variety of network technologies, including military single protocol in the architecture, to be separated into and commercial facilities. The Internet architecture has two layers. TCP provided one particular type of service, been very successful in meeting this goal; it is operated the reliable sequenced data stream, while IP attempted over a wide variety of networks, including long haul to provide a basic building block out of which a variety nets( the ARPANET itself and various X 25 networks of types of service could be built. This building block local area nets (Ethernet, ringnet, etc. ) broadcast was the datagram, which had also been adopted to satellite nets (the DARPa Atlantic Satellite support survivability. Since the reliability associated Network,operating at 64 kilobits per second and the with the delivery of a datagram was not guaranteed, but DARPA Experimental Wideband Satellite Net, best effort, it was possible to build out of the operating within the United States at 3 megabits per datagram a service that was reliable(by acknowledging second), packet radio networks(the DARPa packet and retransmitting at a higher level) or a service which radio network, as well as an experimental British packet traded reliability for the primitive delay characteristics radio net and a network developed by amateur radio of the underlying network substrate. The User Datagram operators), a variety of serial links, ranging from 1200 ACM SIGCOMM Computer Communication Review
ACM SIGCOMM -4- Computer Communication Review services expected in an operating system (e.g. support for timers.) So XNET was designed to run directly on top of the datagram service provided by Internet. Another service which did not fit TCP was real time delivery of digitized speech, which was needed to support the teleconferencing aspect of command and control applications. In real time digital speech, the primary requirement is not a reliable service, but a service which minimizes and smoothes the delay in the delivery of packets. The application layer is digitizing the analog speech, packetizing the resulting bits, and sending them out across the network on a regular basis. They must arrive at the receiver at a regular basis in order to be converted back to the analog signal. If packets do not arrive when expected, it is impossible to reassemble the signal in real time. A surprising observation about the control of variation in delay is that the most serious source of delay in networks is the mechanism to provide reliable delivery. A typical reliable transport protocol responds to a missing packet by requesting a retransmission and delaying the delivery of any subsequent packets until the lost packet has been retransmitted. It then delivers that packet and all remaining ones in sequence. The delay while this occurs can be many times the round trip delivery time of the net, and may completely disrupt the speech reassembly algorithm. In contrast, it is very easy to cope with an occasional missing packet. The missing speech can simply be replaced by a short period of silence, which in most cases does not impair the intelligibility of the speech to the listening human. If it does, high level error correction can occur, and the listener can ask the speaker to repeat the damaged phrase. It was thus decided, fairly early in the development of the Internet architecture, that more than one transport service would be required, and the architecture must be prepared to tolerate simultaneously transports which wish to constrain reliability, delay, or bandwidth, at a minimum. This goal caused TCP and IP, which originally had been a single protocol in the architecture, to be separated into two layers. TCP provided one particular type of service, the reliable sequenced data stream, while IP attempted to provide a basic building block out of which a variety of types of service could be built. This building block was the datagram, which had also been adopted to support survivability. Since the reliability associated with the delivery of a datagram was not guaranteed, but "best effort," it was possible to build out of the datagram a service that was reliable (by acknowledging and retransmitting at a higher level), or a service which traded reliability for the primitive delay characteristics of the underlying network substrate. The User Datagram Protocol (UDP)13 was created to provide a applicationlevel interface to the basic datagram service of Internet. The architecture did not wish to assume that the underlying networks themselves support multiple types of services, because this would violate the goal of using existing networks. Instead, the hope was that multiple types of service could be constructed out of the basic datagram building block using algorithms within the host and the gateway. For example, (although this is not done in most current implementations) it is possible to take datagrams which are associated with a controlled delay but unreliable service and place them at the head of the transmission queues unless their lifetime has expired, in which case they would be discarded; while packets associated with reliable streams would be placed at the back of the queues, but never discarded, no matter how long they had been in the net. It proved more difficult than first hoped to provide multiple types of service without explicit support from the underlying networks. The most serious problem was that networks designed with one particular type of service in mind were not flexible enough to support other services. Most commonly, a network will have been designed under the assumption that it should deliver reliable service, and will inject delays as a part of producing reliable service, whether or not this reliability is desired. The interface behavior defined by X.25, for example, implies reliable delivery, and there is no way to turn this feature off. Therefore, although Internet operates successfully over X.25 networks it cannot deliver the desired variability of type service in that context. Other networks which have an intrinsic datagram service are much more flexible in the type of service they will permit, but these networks are much less common, especially in the long-haul context. 6. Varieties of Networks It was very important for the success of the Internet architecture that it be able to incorporate and utilize a wide variety of network technologies, including military and commercial facilities. The Internet architecture has been very successful in meeting this goal; it is operated over a wide variety of networks, including long haul nets (the ARPANET itself and various X.25 networks), local area nets (Ethernet, ringnet, etc.), broadcast satellite nets (the DARPA Atlantic Satellite Network14,15 operating at 64 kilobits per second and the DARPA Experimental Wideband Satellite Net,16 operating within the United States at 3 megabits per second), packet radio networks (the DARPA packet radio network, as well as an experimental British packet radio net and a network developed by amateur radio operators), a variety of serial links, ranging from 1200
bit per second asynchronous connections to TI links, various organizations which manage the gateways are and a variety of other ad hoc facilities, including not necessarily the same organizations that manage the intercomputer busses and the transport service provided networks to which the gateways are att by the higher layers of other network suites, such as IBMS HASP On the other hand, some of the most significant problems with the Internet today relate to lack of The Internet architecture achieves this flexibility by sufficient tools for distributed management, especially making a minimum set of assumptions about the in the area of routing. In the large internet being function which the net will provide. The basic routing decisions need to be assumption is that network can transport a packet or constrained by policies for resource usage. Today this datagram. The packet must be of reasonable size, can be done only in a very limited way, which requires perhaps 100 bytes minimum, and should be delivered manual setting of tables. This is error-prone and at the with reasonable but not perfect reliability. The network same time not sufficiently powerful. The most important than a point to point l orm of addressing if it is more change in the Internet architecture over the next few years will probably be the development of a new There are a number of services which are explicitly not generation of tools for management of resources in the sumed from the network. These include reliable or context of multiple administrations sequenced delivery, network level broadcast or It is clear that in certain circumstances the Internet priority rank ing of transmitted packet, architecture does not produce ffective a support for multiple types of service, and internal utilization of expensive communication resources as a knowledge of failures, speeds, or delays. If these more tailored architecture would. The headers of services had been required, then in order to Internet packets are fairly long(a typical header is 40 accommodate a network within the Internet, it would be bytes), and if short packets are sent, this overhead is necessary either that the network support these services apparent. The worse case, of course, is the single rectly, or that the network interface software provide character remote login packets, which carry 40 bytes of enhancements to simulate these services at the endpoint header and one byte of data. Actually, it is very difficult of the network. It was felt that this was an undesirable for any protocol suite to claim that these sorts of approach, because these services would have to be re- interchanges are carried out with reasonable efficiency engineered and reimplemented for every single network At the other extreme, large packets for file transfer, with and every single host interface to every network. By perhaps 1, 000 bytes of data, have an overhead for the engineering these services at the transport, for example header of only four percent reliable delivery via TCP, the engineering must be done only once, and the implementation must be done only Another possible source of inefficiency is once for each host. After that, the implementation of retransmission of lost packets. Since Internet does not interface software for a new network is usually very insist that lost packets be recovered at the network level, it may be necessary to retransmit a lost packet rom one end of the internet to the other. This means 7 Other Goals nat the retransmitted packet may cross several intervening nets a second time, whereas recovery at th The three goals discussed so far were those which had network level would not generate this repeat traffic the most profound impact on the design on the This is an example of the tradeoff resulting from the architecture. The remaining goals, because they were decision, discussed above, of providing services from lower in importance, were perhaps less effectively met, the end-points. The network interface code is much or not so completely engineered. The goal of permitting simpler, but the overall efficiency is potentially less distributed management of the Internet has certainly However, if the retransmission rate is low enough(for been met in certain respects. For example, not all of the example, 1%)then the incremental cost is tolerable. As gateways in the Internet are implemented and managed a rough rule of thumb for networks incorporated into by the same agency. There are several different the architecture, a loss of one packet in a hundred is management centers within the deployed Internet, each quite reasonable, but a loss of one packet in ten suggests operating a subset of the gateways, and there is a two- that reliability enhancements be added to the network if tiered routing algorithm which permits gateways from that type of service is required different administrations to exchange routing tables The cost of attaching a host to the Internet is perhaps even though they do not completely trust each other somewhat higher than in other architectures because all and a variety of private routing algorithms used among of the mechanisms to provide the desired types of the gateways in a single administration. Similarly, the service. such as acknowledgments and retransmission ACM SIGCOMM Computer Communication Review
ACM SIGCOMM -5- Computer Communication Review bit per second asynchronous connections to T1 links, and a variety of other ad hoc facilities, including intercomputer busses and the transport service provided by the higher layers of other network suites, such as IBM’s HASP. The Internet architecture achieves this flexibility by making a minimum set of assumptions about the function which the net will provide. The basic assumption is that network can transport a packet or datagram. The packet must be of reasonable size, perhaps 100 bytes minimum, and should be delivered with reasonable but not perfect reliability. The network must have some suitable form of addressing if it is more than a point to point link. There are a number of services which are explicitly not assumed from the network. These include reliable or sequenced delivery, network level broadcast or multicast, priority ranking of transmitted packet, support for multiple types of service, and internal knowledge of failures, speeds, or delays. If these services had been required, then in order to accommodate a network within the Internet, it would be necessary either that the network support these services directly, or that the network interface software provide enhancements to simulate these services at the endpoint of the network. It was felt that this was an undesirable approach, because these services would have to be reengineered and reimplemented for every single network and every single host interface to every network. By engineering these services at the transport, for example reliable delivery via TCP, the engineering must be done only once, and the implementation must be done only once for each host. After that, the implementation of interface software for a new network is usually very simple. 7. Other Goals The three goals discussed so far were those which had the most profound impact on the design on the architecture. The remaining goals, because they were lower in importance, were perhaps less effectively met, or not so completely engineered. The goal of permitting distributed management of the Internet has certainly been met in certain respects. For example, not all of the gateways in the Internet are implemented and managed by the same agency. There are several different management centers within the deployed Internet, each operating a subset of the gateways, and there is a twotiered routing algorithm which permits gateways from different administrations to exchange routing tables, even though they do not completely trust each other, and a variety of private routing algorithms used among the gateways in a single administration. Similarly, the various organizations which manage the gateways are not necessarily the same organizations that manage the networks to which the gateways are attached. On the other hand, some of the most significant problems with the Internet today relate to lack of sufficient tools for distributed management, especially in the area of routing. In the large internet being currently operated, routing decisions need to be constrained by policies for resource usage. Today this can be done only in a very limited way, which requires manual setting of tables. This is error-prone and at the same time not sufficiently powerful. The most important change in the Internet architecture over the next few years will probably be the development of a new generation of tools for management of resources in the context of multiple administrations. It is clear that in certain circumstances, the Internet architecture does not produce as cost effective a utilization of expensive communication resources as a more tailored architecture would. The headers of Internet packets are fairly long (a typical header is 40 bytes), and if short packets are sent, this overhead is apparent. The worse case, of course, is the single character remote login packets, which carry 40 bytes of header and one byte of data. Actually, it is very difficult for any protocol suite to claim that these sorts of interchanges are carried out with reasonable efficiency. At the other extreme, large packets for file transfer, with perhaps 1,000 bytes of data, have an overhead for the header of only four percent. Another possible source of inefficiency is retransmission of lost packets. Since Internet does not insist that lost packets be recovered at the network level, it may be necessary to retransmit a lost packet from one end of the Internet to the other. This means that the retransmitted packet may cross several intervening nets a second time, whereas recovery at the network level would not generate this repeat traffic. This is an example of the tradeoff resulting from the decision, discussed above, of providing services from the end-points. The network interface code is much simpler, but the overall efficiency is potentially less. However, if the retransmission rate is low enough (for example, 1%) then the incremental cost is tolerable. As a rough rule of thumb for networks incorporated into the architecture, a loss of one packet in a hundred is quite reasonable, but a loss of one packet in ten suggests that reliability enhancements be added to the network if that type of service is required. The cost of attaching a host to the Internet is perhaps somewhat higher than in other architectures, because all of the mechanisms to provide the desired types of service, such as acknowledgments and retransmission