4 Traffic Engineering for Multi-service IP Networks Traffic engineering is a set of techniques and tools that can be used to optimize the performance of an operational IP network. An overarching concept, traffic engineering, broadly interpreted, includes other topics discussed in this chapter: routing control mechanisms such as MPLS, and an advanced configuration in the form of policy-based management. These techniques can also be used without deploying the complete traffic engineering frame- work, but their optimal use benefits from it. For example, we shall see that most efficient use of routing control is possible with proper measurement technologies in the network. Traffic engineering is also a cornerstone for the management of a multi-service IP network in a cost-efficient manner. Supporting services in the network in a commercial environment requires that the status of the network is monitored, optimized and controlled. The traffic engineering framework uses service level management and monitoring techniques explained later in Chapters 6 and 7. Especially in the presence of growing traffic volumes, the effec- tiveness of traffic engineering also has implications on network planning and – when used – admission control decisions, which is why it can be said that traffic engineering also lays the foun- dation for the application of techniques in Chapter 5 and Chap- ter 8. Finally, traffic engineering typically makes use of the service Implementing Service Quality in IP Networks Vilho R ¨ ais ¨ anen  2003 John Wiley & Sons, Ltd ISBN: 0-470-84793-X 98 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS quality support mechanisms described in Chapter 3. The relation to end user service quality discussed in Chapter 2 is that ideally traffic engineering is a set of network-side techniques providing engineered quality support level for individual services, as the name of the concept implies. 4.1 TRAFFIC ENGINEERING The traffic engineering (TE) overview [RFC3272] of IETF’s Traffic Engineering working group (tewg) is a good description of the typical traffic engineering tasks, even though true multi-service support is only finding its way to the Internet. IETF’s traffic engi- neering overview is used as a frame of reference for the rest of the topics in this chapter. The TE overview draft defines traffic engineering as the part of network engineering concerned with performance evaluation and performance optimization issues. Further, traffic engineering is defined [RFC3272] to include the application of technology and scientific principles to the measurement, characterization, modelling, and control of Internet traffic. An important area is the enhancement of performance of oper- ational networks, both in terms of resource usage and from the viewpoint of traffic requirements. The obvious goal is to imple- ment the service quality support with as small an amount of valuable network resources as possible. Examples of the resources involved include leased link bandwidth, CPU usage, and buffer space. Routing control is singled out as a means of controlling resource use over-multiple hops. Compared with traditional IP routing protocol-based methods, the target of traffic engineering- oriented routing control can be network-wide traffic distribution. Traffic requirements, manifested to the end user in the form of “emergent properties” of network performance, are to be opti- mized subject to the constraints of the economic realm. Emergent properties presumably correspond roughly to the end user quality experienced discussed in Chapter 2. Further, traffic engineering is intended to help in identifying and structuring goals and priori- ties in enhancing the end user service experience. Reformulated, 4.1 TRAFFIC ENGINEERING 99 traffic engineering is a systematic means of analysing the network state, drawing conclusions from it, and potentially performing a network configuration. As is often the case with complex real- time systems, creation of proper procedures to cope with data acquisition and control procedures is at least half the work. Traffic engineering is perceived as important already in the present-day best-effort Internet, and it is even more so in the multi-service Internet of tomorrow that is in the making at the moment. Some reasons leading to the need for parameter optimiza- tion are: • change in local traffic volume, such as addition of a new major customer in a PoP; • change in the resourcing situation, such as change in provider for leased line capacity; • change in traffic aggregate distribution (locally or globally), for example, due to adoption of a new service type; • exceptional conditions such as malfunctioning of a router. Other focuses areas of traffic engineering are improvement and maintenance of reliable network operations in the face of excep- tional situations, for example, resulting from equipment failure. This requires the ability to detect network fault conditions and the ability to act based on this information. In the case of a multi- service network, fault conditions pertain to a service quality sup- port mechanism in the network. Related definitions and proce- dures for the former will be discussed in Chapter 7. Fallback to back-up route for a traffic aggregate in case of detected link fail- ure is an example of an action taken as a result of detecting an exceptional condition in a network. In what follows, we shall take a look at the context of traffic engi- neering, the traffic engineering process, acquiring information from the network, analysing this information, and means of enhancing performance in a DiffServ network. The discussion is loosely based on a recent RFC on traffic engineering [RFC3272], but is amended for application in a multi-service network as appropriate. A sum- mary of service quality support aggregate level measurement meth- ods has been added. 100 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS 4.1.1 Context of traffic engineering The appropriate methodology used by traffic engineering can be derived from the context of traffic engineering [RFC3272]. The entire context of Internet traffic engineering is defined to include the following four sub-contexts: • Network context defining the different situations in which traffic engineering needs to be applied. The network context includes network structure, network policies, network characteristics, net- work constraints, network quality attributes, and network opti- mization criteria. In a multi-service IP network, the additional task of creating and managing adequate set of service quality support classes is needed. • Problem context defines the concrete issues traffic engineering is applied to. Problem context includes the identification, abstrac- tion of relevant features, representation, formulation, specifi- cation of solution requirements, and specification of desirable features for the solutions. Congestion is named as an example of a typical problem to be solved on the traditional Internet; in a multi-service IP network, sub-optimal performance of a service quality support class could be a further problem. • Solution context defines how issues defined by problem con- text are addressed. Solution context includes analysis, evalua- tion of alternatives, prescription, and resolution. Examples of solving the congestion problem at different timescales, reac- tive/proactive as well as supply/demand alternatives are pro- vided. In principle this phase is similar for best-effort Internet and multi-service IP network, being only more complex in the latter case. • Implementation and operational context defines the methodological application of solution context, including planning, organiza- tion, and execution. The network context must be able to express the essential factors of traffic engineering in such a way that the problem context can be applied to it. A multi-service network with heterogeneous service quality support requirements can be represented as an aggregation of network resources, traffic distribution offered, and the service quality support mechanisms. Network resources include network elements and the capacity and topology of the interconnecting 4.1 TRAFFIC ENGINEERING 101 network. The properties and configurability of network resources are important. Service quality support mechanisms in a DiffServ network include edge router functions, core router functions, and optionally also routing control and admission control means. The inter-operation of different service quality support mechanisms is part of the network context, too. Specific to a DiffServ-based multi-service network, the effectiveness of network utilization can be taken to be part of the network context. Problem context addresses abstraction, representation, and mea- surement of network status in a manner that is relevant for traf- fic engineering. The problem to be solved needs to be explicitly defined, some kind of acceptance criteria must be defined for good solutions, and requirements for solution context should be defined. The requirements, in general, can relate to the following: • service quality support aggregates; • measurement methods; • analysis methods; • network reconfiguration means. The different factors in the above list are often interdependent. For example, unless there is an efficient means of reconfiguring the network, there is less benefit in making detailed analyses of the network state. The network status measurement may encompass multiple levels of abstraction, ranging from individual resources to Autonomous Domain (AD)-wide representations. We shall return to this issue in Chapters 6 and 7. The development of suitable performance opti- mization means is an important issue. For a multi-service network in general, service quality support aggregate characteristics such as delays and packet loss measures are important. On the network level, the utilization level of the network is vital. Examples of solution context include specific policies, techniques and methodologies, parameters, and guidelines that are imple- mented to perform the task specified within the problem context. The application of measurement tools and methodologies, as well as routing control, are listed as typical important solution con- text components. Implementation and operative context are more administrative, organizational, and executive in nature than the previous steps. This 102 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS aspect of traffic engineering is not central to this book, but some related aspects will pop up sporadically in the following chapters. The authors of [RFC3272] further note that the application of the traffic engineering context – especially with congestion – can be anticipatory or corrective. In the former case, the choice of good measurement, analysis, and performance optimization tools is especially important. 4.1.2 The traffic engineering process The traffic engineering process is defined [RFC3272] as consisting of the following parts: • Definition of relevant control policies. This can be considered as being outside of the actual traffic engineering loop, controlling its progress. Naturally control policies can be and are adjusted based on observed network performance. • Feedback mechanisms for acquiring measurement data from pro- duction network. • Analysis of network state and characterization of traffic workload. • Performance optimization of the network. Thetrafficengineering processisiterative, as illustratedin Figure 4.1. For readers familiar with the philosophy of science, the picture is reminiscent of the idealized depiction of the induction/deduction sequence in science [Old86]. An example of application of traffic engineering in a large-scale production network can be found in [FGL + 00]. The first phase is the definition of the control policies. Inputs used for devising these may include the required service quality support of different traffic types – related to business models of the network operator – and optimization algorithms. For example, the network may have a busy hour configuration as well as an off-peak hour configuration, which are different from each other. Some service quality support aggregates can be defined as more important than others when the triggering and execution of cor- rective actions are being considered. According to [RFC3272], feedback from the network – the sec- ond stage in traffic engineering process – does not have to be limited to data obtained from the network only, but can also 4.1 TRAFFIC ENGINEERING 103 Definition of control policies OptimizeAnalyse Feedback from network Network Measure Configure Figure 4.1 Traffic engineering process of [RFC3272]  International Tele- communication Union use synthetic workloads in case immediate measurement results are not available. Synthetization of measurement results can be based on extrapolation from earlier data, for instance. In the case of a multi-service network, synthetization of performance data is often more challenging than for a single-service network, but this scheme may be necessary to better understand the status of the network. We shall discuss measurement means briefly in Section 4.1.3 and on a more general level in Chapter 7. Analysis can be either reactive or proactive. In reactive mode, analysis attempts to identify possible locations in the network hav- ing sub-optimal performance, to trace the root cause, and may also test different ways of solving the problem. In proactive mode, the goal is to identify optimization targets to prevent future per- formance problems. The proactive mode is often important in understanding and anticipating the effect of traffic volume growth and changing traffic mix to the configuration of a multi-service IP network. Possible tools for performing analysis include mod- elling, simulation, and traffic matrices. Traffic analysis does not have to produce the actual solution, but can also be limited to a descriptive mode. The optimization phase includes the means of selecting the actual method to be used for optimizing network performance. In this phase, the network operator can greatly benefit from using a proactive network performance analysis mode, if the analysis means are good enough. Optimally, the analysis machinery 104 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS can be used for benchmarking potential network configurations in advance. Automation of the above process is a challenge in the general case. It requires finding the correct optimization tasks to be auto- mated, and defining a correct interface for human supervision and intervention. At the same time, instabilities due to control mecha- nisms must be avoided. 4.1.3 Obtaining performance data from the network and analysing it Feedback from the network can be obtained from multiple sources and on different levels of abstraction (see Figure 4.2). Starting with the most detailed method, individual network elements can be queried for information. Network elements may also provide notification mechanisms for informing the network management system of predefined conditions transpiring in the network. In some cases, individual network elements may have information about a larger part of the network than the network element itself. An example of this is the routing table of a router running a Link State type routing protocol such as OSPF. For most characteristics, obtaining information on higher abstrac- tion level than that of individual network elements is made possible by a Network Management System (NMS). Forming of higher-level characteristics can be achieved, for example, by applying thresh- olding to information collected from individual network elements, as will be discussed in Chapter 7. To give an example, with such a method one can see the most heavily loaded routers in a net- work domain. The management system can typically provide averages and tendency analyses in addition to an abstracted momentary net- work status. For a multi-service network, the performance of the network can be monitored on the level of traffic aggregates for a network domain. Finally, the feedback can be in the form of service quality. This requires a way of estimating service level performance. The abstraction levels are compared in Table 4.1. The subject of performance measurements will be covered more thor- oughly in Chapter 7. Typically, network elements such as routers provide a set of counters accessible through a management interface, for 4.1 TRAFFIC ENGINEERING 105 AD NE NE NE Endpoint AD Endpoint Aggregate performance Service performance Figure 4.2 The relations between performance measures Note : The management system provides information about performance of individual network elements within an AD, whereas aggregate perfor- mance and service performance relate to measurements spanning multiple elements. An AD may have multiple aggregate performance measures, for example, relating to different DSCPs Table 4.1 A comparison of network monitoring levels Monitoring level Typical measured characteristics Per network element Overall load and per- traffic aggregate statistics. IP management system Network-wide data, averages, trend analysis. Aggregate performance Delay, jitter, packet loss, and available bandwidth in anetworkdomain. Service level Service Level Specification components (see Chapter 6). example, Simple Network Management Protocol (SNMP). The contents of the accessible information are defined in Management Information Bases (MIBs). Some routers may use vendor-specific, non-standard interfaces. Two types of measurements relevant to traffic engineering will be discussed in this chapter for purposes of concreteness. Traffic aggregate performance measurements (Section 4.1.3.1) data is rele- vant for optimizing the parameters of a DiffServ network domain. They can also be used as an input for routing control. Traffic load measurements are typically used for routing control and capacity expansion need evaluation purposes. 4.1.3.1 Traffic aggregate performance measurements Aggregate level performance here means network performance of a service quality support aggregate, such as the behavioural aggregate of DiffServ, over multiple IP hops. Thus, they include 106 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS the effect of multiple network elements as a part of the measurement definition. In view of performance data relating to a particular service quality support aggregate, aggregate level performance measurement means may also require less processing than performance estimation obtained by combining data from individual network elements. Generally, three different types of methods can be used for aggregate level performance assessment: 1. passive (non-intrusive) measurements; 2. active (intrusive) measurements; 3. piggybacking measurements. The measurement data obtained by one or more of these mecha- nisms can be processed in different ways to obtain relevant char- acteristics. One example of such processed characteristics is packet loss correlation modelling [Cla01, TIPHON-5] which can be used to estimate the quality of traffic aggregate used for transporting VoIP or multimedia streaming. More generally, ITU-T’s recom- mendation [I.380] lists the following measurable quantities: packet transfer delay, delay variation, packet error ratio, and packet loss ratio. Availability is characterized by the PIA and PIU parameters discussed in Chapter 2. A generic issue related to measurements, proper measurement methodology, including adequate sampling, is very important in obtaining meaningful results [RFC2330, RFC3432]. Measurement results must form a representative sample of network behaviour in order for optimization techniques to be applicable. To this end, the measurement point placement scenarios need to be defined carefully [I.380]. Likewise, the treatment of error conditions such as packets with bit errors need to be defined. Let us take a look at the aggregate performance measurement methods next. 4.1.3.1.1 Passive measurements Passive measurements do not require additional traffic to be injected into the operational network. Instead, the normal traffic in an opera- tional network can be monitored. One or more measurement points canbeusedinasinglenetworkdomainpermonitoredtraffictype. With one measurement point, traffic load and protocol statistics can be monitored. In the case the monitored streams include timestamp- ing and sequence numbering information, which is the case for the RTP, also delay jitter and packet loss measurements can be made, [...]... measurements, when the operating system and TCP /IP stack-related 108 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS properties of the measuring hosts are well known Active measurements have been subject to standardization in IETF’s IP Performance Metrics working group (IPPM WG), ITU-T [I380], and ETSI [TIPHON-5] To give an example of IETF work, the IPPM working group has defined measurement metrics... MULTI-SERVICE IP NETWORKS to inter-domain resource management by triggering renegotiation of SLAs with other network transport providers 4.1.5 Scope of network optimization The scope of IP transport network optimization means is important The target of IP target network optimization mechanism can be, for example, an interface within a router, an individual network element, routing area, or an IP domain,... in case of back-up routes requires extensions to standard IP routing In OSPF, for example, affecting 124 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS routing directly requires a management system interface towards the link state table Using multiple routes for a traffic aggregate is possible with link state protocols such as OSPF, if multiple-path routing is supported by the routing protocol instances... multi-service support on the IP layer and link layer is very important Depending on the topology, routing control is a tool for affecting distribution of traffic in the network For OSPF routing, adjustment of OSPF metrics is a means of controlling routing [FT00] Link layer tunnelling such as MPLS can be used for more advanced tunnelling, multi-protocol support (e.g., IPv4 and IPv6) being possible with... synchronized with the measuring point, and that timestamp marking can be trusted Passive measurements have been subject to standardization at least in IETF [RFC1757, RFC2021, RFC3432] and ETSI’s IP telephony project TIPHON [TIPHON-5] Passive measurements do not add further traffic into the network, but require more processing than active measurements, for example They may also require transfitting of large amounts... and traffic aggregate performance characteristics Obtaining accurate information about traffic load in IP networks where routing is based purely on IP routing protocols is important mainly because of two factors: the non-connection-oriented nature of the Internet Protocol, and the distributed nature of IP routing protocols As a result of these two factors, traffic volumes on a particular link in an AD... ENGINEERING FOR MULTI-SERVICE IP NETWORKS Clearly, proper dimensioning of the network is an important step for QoS-critical services such as VoIP Performing this endto-end is a non-trivial task Architectures and reference models for this are discussed in Chapter 5, and the case study in Chapter 9 includes an example of QoS dimensioning 4.4 SUMMARY Traffic engineering tools for IP networks have been reviewed,... out as a particular challenge If deployed, automated network selfoptimization must not lead to instabilities in network behaviour 114 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS With reference to the discussion about multiplexing gains in the preceding chapter, one goal of traffic engineering in a multi-service network is to maintain the optimal utilization level of network, given the composition... ENGINEERING FOR MULTI-SERVICE IP NETWORKS network performance measurements often have different scope than service level measurements Drawing conclusions of performance optimization actions based on aggregate performance results requires the link between end-toend service performance and aggregate performances to be known One tool for this is the use of QoS budgets [TIPHON-3] A simple application of... methodology With proper measurement packet format, delay, delay jitter, packet loss and packet loss correlation performance for a traffic aggregate can be directly measured with active measurements [TIPHON-5] Further, the IPPM working group is in the process of defining a reordering metric Furthermore, available bottleneck bandwidth for the traffic aggregate in question can be estimated using the packet pair method . of DiffServ, over multiple IP hops. Thus, they include 106 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS the effect of multiple network elements as. standardization in IETF’s IP Per- formance Metrics working group (IPPM WG), ITU-T [I380], and ETSI [TIPHON-5]. To give an example of IETF work, the IPPM working group