study of proposed internet congestion control mechanisms potx

Retransmission Rat Chste Anas for Time Period One ‘Sample Plot Anarsng the Iftsence of Condition and Congestion Control Algol ‘Summary of StattallySigniieant Oullies a Tine Period One Fi

Study of Proposed Internet

Congestion Control Mechanisms

Kevin L Mills James 1, Filiben Dong Yeon Cho Edward Schwartz Daniel Genin

INUSTP wien insite o stondors ond thst ma

Study of Proposed Internet

Congestion Control Mechanisms

Kevin L Mis James J Flliben Dong Yeon Cho edward Sehwarte Information Technology Laboratory

May 2010

National laste of Sana and Toso iưenG to gly at he

National Institute of Standards and Technology Special Publication 500-282

spec Publ 500-282, 534 pages (May 2010)

‘CODEN: NSPUE2

U.S, GOVERNMENT PRINTING OFFICE

WASHINGTON: 2010 For sale by the Superintendent of Documents, U.S Government Printing Office Intemet: bookstore gpo.gov — Phone: (202) 512-1800 — Fax: (202) 512-2250

Acknowledgments

The foundation fr ths work began in 200, when Jan Yuan, now an associate professor

in the department of electronic engineering at Tsinghea University, rived at the National stinte for Standards and Technology (NIST) 0 begin four years of collaboration with Kevin Mills, Dr Yuan introduced to NIST the tea of viewing large ommnication networks as complex systems, and worked together with Dr Mills #0 explore the implications of investigating the Internets acomplen system, During that time, Dr, Yuan developed acellular automata model ofan Intere-tike network, which served asthe bass from which MesoNet was crested MesoNet isthe reduced parameter diserete-event simulator wed inthe experiments reported in his sty The collaboration between Dis, Yuan and Mills edt ceaion of two proposals: (1) to develop a project on tmeasurement science for complex information systems and (2) to formulate a complex Systems research program within the Information Technology Laboratory (ITL) at NIST

1m 2006, Dr Mills submited a projet proposal io an internal NIST competion, known as innovations in measurement sence (IMS), aig wo gain approval fom the NIST director to apply measurement science to complex information systems Wiliam Jefe, the NIST diector at tat ime, supposed the project proposal hich also found favor with Cita Furlan, director of ITL In parallel, a group of FTL researchers, including Chris Dabrowski, Fern Hurt, Viaini Marbukh and Kevin Mills proposed the eration ofan TTL program to study complex systems in general This second proposal was well received by the management team within TTL, which founded a complex systems (CxS) program and recruited Sandy Ressler as the program manager Sandy has provided Seas supporto the work documented in this report In Fc, he added esources wo the project so that we could explore the use of interactive visalztion a8 a tol to reveal obal behavior in lage distributed systems, Further, he regularly attended the biweekly Project meetings and provided many usetul suggestions for improving the content and Presentation ofthe work In 2007, while we were exploring potential challenge problems fora measurement science of eomplex systems, Viaiit Marokh suggested that we consider the collection

Of congestion contol algorithms being_proposed io augment or replace congestion contol procedures within the standard transmission control protocol (TCP), which opertes"wihin every computer connected to the Inlerel, Ultimately, Vladimir's Suggestion led to the study documented inthis epon Through the Inlerel Congestion Contto! Research Group (ICCRG) of the Inert Research Task Force ORTH), we contacted the community of researchers involved in developing fuure congestion contol

‘mechanisms for the ltemel Several members of the ICCRG had perforined empirical Suis of elected congestion contol algorithms operating within a small topology

‘These empirical reslls proved valuable in verifying our models of individual congestion contol algorithms, Wesley Edy and Michael Weld, o-cts of the ICCRG, encouraged us to share our work withthe group and provided ample agenda time at several meetings for us 10 Fist present our methods and later out findings, We aso appreciate the general support for our work from researchers within the ICCRG community, as well as several researchers within the Transport Modeling Research Group (TMG) of the IRTF

Mills, et al Special Publication 500-282 iit

In addition wo support from Internet researchers, we also benefited from the work

of several colleagues who made temporary visits to NIST For example, Britany Devine worked on the project for thee months under the Summer University Research Fellowship (SURF) program administered by the National Science Foundation (NSF) Brittany contributed prototype code to make measurements on flow groups as explained

Jn Chapter 8 Edward Sehwartz, one ofthe study authors, also worked on the project for three months under the SURF program Ed, now a doctoral student at Camegie Mellon University, developed the inital models of CTCP, FAST, HSTCP, H-TCP and Scalable TCP that were used within this study Ed also conducted some preliminary experiments comparing CTCP, FAST and standard TCP under select scenarios Cedtie Houard contributed two years of work (0 the project as a guest scientist visiting from France Specifically, Cédric developed DiVisa, an interactive system for visualizing

‘multidimensional data

Supported by these many valuable contributions, the authors produced the current report t© document a set of methods, models, experiments and findings, which were produced largely over the period of 2006-2009 ‘The study encompasses a wide scope and substantial breadth, which could not he documented completely in less than $00 pages

We are indebted to Christopher Dabrowski and Stefan Leigh, two NIST colleagues who

‘made careful, independent readings of the entie report, and who also provided many useful suggestions for improving the manuscript The review provided by Dabrowski and Leigh upheld the highest standards of NIST requirements for intemal review prior to publication OF course, in a report of such length, scope and depth, erors undoubtedly remain Responsibility for such residual errors lies squarely atthe feet of the authors

Not: the document cover and chapter dividers contain images generated fom the sty tex with aplleesesarllened

Executive Summary

When first introduced in the early 1980s the Intemet appeared to be an interesting engineering curiosity, providing resource sharing and data communication services t6 support scientific researchers Beginning with the introduction of the World Wide Web (circa 1995), the fundamental data communication services provided by the Internet transformed into a global infrastructure for commerce, education and entertainment, Later developments (cca 2005) built upon so-called Web Services 10 provide innovative social networking technologies that citizens the world over ean use t0 organize and collaborate around collective interests Along the way innovative cell phones and other handheld devices were introduced and integrated with the Internet and the Web wo extend available information and interaction services to people anylime, anywhere The future promises globe interconnected by large, distributed information networks, where people routinely interact in new and unexpected ways Realizing this future relies in large part

‘on our ability to understand and engineer globally distributed systems of interconnected hardware and software components, including their use by people At present, society is technologically capable of building such systems but lacks the fundamental knowledge required to understand and predict macroscopic behaviors that may arise from complex interactions as such systems evolve with the addition of new technologies and new patterns of use A similar lack of Knowledge may impede progress with respect to other large systems engineered by society For these reasons, researchers in the Information

‘Technology Laboratory (ITL) of the National Instiure of Standards and Technology (NIST) embarked upon a measurement science based program of research for complex systems, The NIST Complex Systems Program aims fo investigate and evaluate methods and tools that system designers might adopt to improve scientific understanding of large Aistributed systems, such as information networks, eletrc grids and transportation webs

‘As part of the NIST Complex Systems Program, this special publication investigates and evaluates modeling and analysis methods that can be applied to predict and understand macroscopic behavior and variations in user experience that may arise as engineers introduce changes in software components into a large information network, such a6 the Internet The Intemet consists of millions (Someday billions) of interconnected components that may be changed independently For example, everytime vendors of major operating systems introduce software updates, millions of users download nev software modules into computers connected to the Internet, As another example, users may download software to support new functions, such as social networking or distributed gaming At the current state of the at, system designers lack techniques to predict global behaviors that may arise in the Interet as a result of interactions among existing and allered software components, Similarly hardware faults and unexpected usage patterns may occur within the Internet Engineers have insufficient melhods and tools available 10 forecast global behaviors and resulting effects on Individual users The study described inthis special publication aims to improve existing knowledge about a range of methods and tools that could be applied to understand and predict behavior in such complex information systems To give our study a concrete context, we selected a challenging problem of current interest and relevance for the Internet at large Specifically we study the likely consequences for macroscopic behavior and For individual users should any of several

proposed mechanisms be introduced to augment or seplace congestion conteol procedures

ân the standard transmission control protocol (TCP), which is currently deployed 10 regulate the rate of information tansfer among computers conaected 10 the Interne Congestion contol procedures allow individual computers to measure available capacity

‘on network paths and to attempt to transfer information as quickly as possible Because conditions vay with time, congestion contol procedures also enable detection of congestion that may arise a$ too many computers atempt 10 use a network path Upon detecting congestion, TCP first substantially slows a computer’ rate of transfer and then tempts to slowly inerease the rate Researchers have predicted thatthe standard TCP congestion control procedures will inhibit users trom realizing increased transfer speeds

as capacity expands in the Internet backbone from the current rate of 10 Gigabits per second (Gbps) 10 100 Gbps and beyond For this reason, various researchers have proposed changes to the congestion control procedures implemented in standard TCP, At the current state ofthe at, such proposed changes have been studied on individual long- lived flows using analytical methods and also studied using simulation and! empirical

‘measurements in small topologies with limited types of data trafic, Though researchers and engineers would like to predict the effects of such changes on macroscopie behavior and on individual users, no techniques are currently available to make such extrapolations

‘o large, fast topologies transporting hundreds of thousands of simultaneous data transfers

of various sizes under a wide range of network conditions The study documented in this special publication describes and evaluates modeling and analysis techniques applied t0 make such extrapolations for seven proposed alternatives to standard TCP congestion contol procedures

We apply techniques often used by scientists at NIST when studying physical systems First, we propose an abstract simulation model, representing a data communications network (including TCP) with only 20 parameters, as compared with the hundreds of parameters typically used in detailed Internet simulators Second, we adopt 2-level-per-facior experimental designs, which consider each parameter at only two Values, 88 compared with the billion or so values that each parameter could possibly tke

on, Third, we leverage orthogonal fractional factorial (OFF) experiment designs that tenable us to model a sparse but balanced set of parameter combinations spread widely thoughout the space of possible combinations, Reducing the number of parameters, parameter levels and combinations enables feasible simulation of large networks under a

‘wide range of conditions, Thitd, we use a vatiety of statistical analysis and visualization techniques designed to explore multidimensional data sets Fourth, we use detailed analyses of time series as required to supplement findings from statistical analyses We

‘demonstrate that our proposed combination of modeling and analysis techniques allows

us to prediet the influence of seven proposed congestion control mechanisms on macroscopic network behavior and individual user experience,

In summary, this special publication contributes to current knowledge about

‘modeling and analysis techniques for complex information systems and also contributes {0 the body of knowiedge surrounding proposals for improving congestion contol mechanisms considered for deployment in the Inteme Six specific contributions improve current knowledge regarding techniques to understand and prediet behavior in complex information systems First, we summarize the current state of the art in

‘modeling and analysis of communication networks and we identify several hard problems

that inhibit the study of large, fast networks Second, we propose an approach to construct simulation models witha reduced parameter space As a corollary contribution,

‘we identify and explore some alemative, promising modeling approaches, including uid flow models and hybrid models, which combine quantized time calculations with diserete events Third, we describe and demonstrate how tao-level OFF experiment designs ean be applied to reduce the number of parameter combinations that must be considered, while yielding maximum information from available simulation resources Fourth, we describe and apply a variety of analysis and visualization techniques for interpreting multidimensional data, We first use these techniques to conduct sensitivity analyses of out simulation model and then apply the techniques to compare congestion

‘control mechanisms Fifth, we evaluate our proposed modeling and analysis techniques, discussing the strengths and weaknesses of various methods and identifying those methods that proved most effective for our study, Sixth, we outline Future research needed to improve upon the methods we evalusted Our six contributions enhance understanding of methods and tools availabe to designers of complex systems

Four specific contributions add tothe body of knowledge surrounding proposals for improving Internet congestion control First, we characterize likely macroscopic behavior and user performance for seven proposed alternatives to TCP congestion contol procedures In doing so, we reveal that proposed improvements to TCP congestion control would benefit individual users under a specific combination of circumstances unlikely to arise very often in the general Intemet We also identify some cautionary findings with respect 19 various congestion control mechanisms we study Second, we identify key behavioral characteristics to be considered when comparing congestion

‘control mechanisms We found these characteristics through analyses of experiment data, rather than through a priori analyses Part of our method was to collect as much measurement data as possible and then (0 use statistical techniques (e-., correlation and principal components analyses) to identity those measures representing different facets of system behavior Then, given selected measures, we could determine the key factors influencing macroscopie behavior and user experience Previous studies of congestion

‘control mechanisms did not reflect these Key factors Third, we iemify and compare the

‘main differences among the congestion control mechanisms we studied, We show that, for the key behavioral factors we identify one ofthe seven mechanisms we studied fares better than the others Fourth, we suggest some future research directions related to Iniemet congestion control Our four contributions should help researchers to better understand the problem space surrounding congestion contol inthe Internet

While the current study is quite comprehensive with respect to the study of large distributed systems, we have certainly not covered every method and technique that might prove useful For example, a related project in the NIST Complex Systems Program is investigating how Markov models, coupled with perturbation analysis, Ejgenanalysis and graph theory, canbe used to identify specific aspects of system designs that might significantly degrade performance when subjected to failures Further, while some of the methods we applied appear quite effective in the context of Intemet congestion control, we also need to demonstrat effectiveness in other applications In summary, this study makes substantial contributions to methods for modeling and analyzing complex information systems and also provides significant information to the

‘community of researchers studying Internet congestion control

222 Expanded Simulation Studies

233 Expanded Anal Studs 2.3 Seleted Approach

24 Mara Problem ‘4 Mel Sale —-

343 Mel Validation

243 Tractable Anata

244 Causal Analyste

25 Seleted Slaions aed Posse

"ES Scale Reduction 3⁄53 Sensiity Analysis and Rey Empirical Comparisons

253 Statstial Analysis Methods

254 Data Supported Domain Experi

253 Domain faperthe and Incremental Design

nS sus aie

32 Network Topolegs “S121 Fourier Sroctre

3122 Heterogeneous

3123 Che XI34 EnelPah Naulne

3135 Shaated Abilene Characteristic

33 Simlated Packets [XL Relating Abstract Tim to Real Tine

321 Simulation Control Parameters

322 Parameters Defining User Bhavior

4323, Parameters Adapting Network Topology

‘324 Parameters Deserbing Sourees and Receiver

325 Parameters Spctsng Special LongLived Flows

326 Parameters for Transport Protocol

327 Parameters Menying Monitored Links

328 Reporting Parameter Slings

‘SA61 Mcasorements Common o All Rowtrs

3362 Mensarements Unique to Acces Rosters

54 Performance Properties of MesoNer

SSL Procexing Requirements SEA2 Memory Requirements

AON VY Seater Mots

4132, Correlation Computations

4135 Combined Mates Vitalin

44134 ndesctndes Plt

4414 Principat Components Ama

415 10Ste Graphieat Anayss of Sled Responses {Lia Other xpiratry Pts and Analyses

442 espeiment Design for MesoNet Semstiiy Ana

CET" MeoNet Factor ‘Sinufation Control Parameters

„

”

4214 Parameters Altering Properties of Sources ind Receivers 9L

4213 Parameters Controliog Source Startyp Pllern 8

4216 Parameters Related to TCP Operation 93 42.17 Summary ot Factors Sleted for Senstivily Anas 93

422 Numb of Levels and Settings for MesoNet Factors M221 ‘Two! FactorSetings CC CC - TT 38 đi

$222 Rationale tr land Ramifcains of Nebrork Fact 422.3 Rationale for and Ramifications of User Faeior Sets — 96 422.4 Rationale Factor Setigs for land Remifcations of Source & Receiver ae”

4225 Raton ar|[and RanificaHonxof) Feulueol Radlor

423 Specie Combinations Simulated »

43 Experiment Fxecaton HST Resoure Requirements or Simulations 100 too

432 Data Collection and Summariation is

44 Correlation Analysis and Chastering 108

45 Principal Components Anal as

46 Senivity Analysis "U6 Semitity Analy Guided by Correlation Anal m mu

‘AGLI Congestion Related Responses ` ng

$612 Delay-Related Responses us 4461.3 Responses Related to Macroscopic Trwoghpat us 446.14 Responses Related to Advanaged Flow Classes us

462 Sensitivity Analysis Guided by Principal Components Anais tạp

4462.3 Throughput for Advantaged Flows Bs

‘4624 Macrscople Through mm

4463 Summary ‘MGI Main Aspects of Findings fom the Sensivity Anal of Model Behavior Bs i

4632 Major Factors Influencing Model 7

4633 Factors Exhibiing Lite fafluence on Model Behavior 129 47° Exploring EMets of Butfer Sizing ATL iets on Delay Variation be tô

443 EeetenOtteFAspecteafNHhonk Bedavne m

Modeling Alternate Congestion Cont

1" TCP Congestion Contra SALI Conmeeton Phi

512 Tranter Phase

Stat Sta

5122 Limited Slow Start 5112.3 Setting Sow Start Thresboe 5.13 ‘Transfer Phane= Congestion Avene

S131” Increae Congestion Window fle Acknowledgment

5132 Decrease Congestion Window after Sig SUBS Decrease ShoStart Threshold and Rest

Mechanisms

Window after Timeout Congestion Avoidance” “ MS

52 Congestion Avoidance Procedure for Six Alsemue Congestion Conical

S21 BIC 4311 Inereae Procedares w us

5233 Timeout Procedures CC CC CC 1M S334 Periodic Procedures lạ

“%3 Mdling the Transfer Phase fn MesoNet 159

"S31 General Data Transfer Procedare 139

EAS Congestion Aveldance tại S331 Acknowledgment Procelurs lại S132 Nga Acknowledgment Procedures lôi S84 Timeout Procedures tại S4 enn Silat Cneston Con Seca 1a

Standard TCP Congestion Contel Model 163 EAR tehor af IC Compson Cont! Ml tes

43 shaver af CICP Congestion Conteo Moda tế

‘544 Behavior af FAST Congestion Contra Model tớ EAS Behavior of HSTCP Congestion Cnttl Mod mm

546 Behavior of HATCP Congestion Control Model mà

547 Behavior of Scalable TCP Congestion Control Model 3 S48 Sommary of ichavir of MewNet Congestion Control Model v8

6 Comparing Congestion Control Regincs i Large, Fast Network mg

‘Experiment (TT Semaison Parameer Desi 183 Hệ

G11 Mesures of Macrscopie Behavior 190

6132 Metouresof User Experience "

6133 Messuresof Waffer Usage 1

62 Experiment Execution and Data Collectio ws

‘G21 Experiment Execution Bế

63 Data Anais Approach

Mills et al Special Publication 800-282 "

4 Time Period One (RP) GALI Chester Anal fr PPE TTT— CC

6412 Condiion Response Summary for TPL

6413 Anal of Significant Responses or T 4.14 Summary of Results for TPL ———

643 ‘Thme Period Two (1P2)

(AI Chướ AalwAES CC CC

4432 Condiiontespome Summary for TPS

6433 Anal of igiicant Response or T 64.34 Summary of Results for TS ————

644 Agengaled RepeneesTAIR)

fet Clester Anas for Fotale ot Resjunee Suramar fr Toa CC ——

4443 Cm 6/443 AnadeafSgifcaml Meganee Tar Totals (6444 Summary of Reals for Total

7 Comparing Congestion Control Regimes ina Sealed-Down Network

TA Experiment Design 7411 Changes in Robustness Factors and Fined Factors

‘Orthogonal Practonal Factorial Design of Roberts Conditions — 71.3 Domnin View of Rabusines Condos

12 Esperiment Execution and Data Cole

73 Data Analysis Approach

7 Resale

TAL Time Period One (MPD)

72411 Che AnalofA tor TPL

72412 Condition Response Summary for TPL TALS Analysis of iglicant Response for TPT 74.14 Summary of Rests for TPL

742 Time Peviod T21 Chester Anal 7422 Condiion Two (1P2), Response Summary for TPE for TPE =

72423 Analysis of igniicant Responses or TPE

7424 Summary of Results for TP2

TAS Te Period Three TPS)

ary a Results or TP ————

TALL Chaser Anas for 1 mm

7432 Condiion Response Summary for TPS 2 TASS Analsiof Significant Response or T 3m 74.34 Summary of Rests for TS 2

744 Agaregnted Responses (Tata) ny

“rt Clester Anas for Tote 25

7442 Condi: Respome Samir for Total 2s

744423 Anal Significant Responses for Totals mg

74444 Summary of Reals for Total

16 Findings "rst Roding WT

753 Fining 2 75A Finding 3

Comparing Congestion Control Regimes in Heterogeneous Network a6

'Robetncs Factors and Fixe Factors ‘ALLL Constraints on Flows of Large Si 37 x0 L2 Fined Experiment Factors mm

12 Orthogonal Fractional Factorial Design si

13 Doman View of Robustness Conditions ms

#14 Responses Measured Fc} R2 Experiment Execution and Data Colt R21 Compating Macrosepie Responses 31 mg

#22 Computing User Experience Response Be

SAL" Analyzing Macroscopic Behavior xe

#32 Analyzing User Byperence a7

SA Relative User Experience m

‘S431 High Ini Sloe-star Threshold mm S482 Low initial Slow-start Threshold 30 ASS Summary of Diferences in Relative G as

9 Comparing Congestion Control Regimes in a Large, Fast, Heterogencous SLI Changesin Experiment Dees Network 85 86

9512 Changes in Orthogonal Fractional Factorial Design of Robustness

9.1.3 Chameci Dommim Vievof RabndnevComiiiome sử Mills et al Special Publication 800-282 say

92

m

m

342 Absate User Experience

9443 Relative User Experience

3 Eulng SS Finding 1 SON TT

TALIA Vay Bounds Inn Saety

2 Characters

A021 BC, A0433 CTCP A0133 FAST A0124 FANEAT ings usted A0126 HÌCP

1027 SehbieTCP 10.13 Recommendations

102 Conclasions about Met

M21 Discrete Even Simlation 1.22 Seale Reduction Techniqacs 102.2 Model Restriction and Parameter Claserng

10222 Twortevel Experiment Designs

10333 Orthogonal Fractional Factorial Experiment (OFF)

10224 Corraion Avalyss and Clustering

10228 Principal Components Analysis (PCA) 102.3 Medel Validation Techniques

WR1 Sendtig Amabxk

10243 Key Empieeal Comparisons

1024 Data Analysis Methods W241 TensStep Graphical Anabam

16242 ChuerAnahnie

10243 Castom Malidisendunl Vaualiatbns

10244 Exploratory Multidimensional Interactive Visualization —

1028 Causality Analysis Methods 102.81” Principal Componsnts Analysis (PCAY

10252, Detaled Messorements M83 Scene Method

10263 Incremental Desga 7 8

M1 Wiipraphy THỊ Fundamental of fern Deis as

112 Standard Trancaison Control Praia! (TCP)

114 Internet Teac Characters ass

1 Researeh on Data Transport in High Spec High Delay Networks hà 1.7 Proposed Replacement Congestion Control Mechanar the Iie se

18 Evaluating Internet Congestion Control Mechanisms as

19 Inernet Simulation 110 Sopporting Software Toa sod Models as ass 1.11 Supporting Statitial Techaiques is

13 General Related Referens LL Analytical internet Modes vã đa

‘Ad Flukdlow Approximation Models đa

11” Madeling Many Flaws on One Link as

2 Uiliy of Fuidtow Approximation Mod ar

13 Limitations of Fsaslow Approximation Mod aes

2 AppvingFledsow Appeotimation Modesto Compare Aersate Congestion

“Experiment Eseeutio and Data Colleton nh

‘Supplementary Seasivity Analysts Ress 3

Experiment Execution and Data Caledon Bs

C32 Principal Components Analy mm C33 Exploratory Analyixofy74332Seatzr PotBinealion Ki

(C41 Cangeston- Related Response 07 CMA2 Delap-Retated Responses 310 C33 Responses Related Lo Mace m C344 Responses Related to Advas Si Suinmary of Findings trom Sensitivity Analysis sis Exploring fects of Baer Sing sie

Mills et al Special Publication 800-282 "

By Appendis D_ 10Step Graphical Analysis Technique s—

‘Sample Data Summarizaton: 22 Responses foreach of 6 Simulation Runs 103

‘Combined Matrix of Setter Plots and Corrlaton Vales fr 22 Responses 103 Frequency Dstebution af the Absolute Vale of Correlations for AM Paes of

ifes Plt for Response }10 (Average Retransmission Rate)

Main-Eets Plot for Response 32 (A¥erage Instantaneous Throughput for VY

Flows Eft Pt for Response 15 (Average Smoothed RowndTrip Tims) us ne

Main-Eets in-Eets Pla for PCS (Throughput for Avantaged Flows) Plt for PC2 (Network Dla

‘Effet Pot fr PCA (Macroscope Thevughpat)

Mult-actor Seater Pt of Smoothed Round Teip Time (15) foreach of he TT

Average Condition Ranking Display on Vericesofa Cube ARS

‘TCP Connection Establishment Procedures Leading to Connection Fallare Hô

‘TCP Conneeton Establishment Procedures Leading to Initation ofthe Trans

Mills et al Special Publication §00-282 sell

“

en

‘Sample Change in Congestion Window over Time snr Standard Slow Start and

‘Sample Change a Congestion Window oer Time under Lined Siow Sar and

Congestion Avoldance

Simulated Dumbbell Topology Tor MesoNet Verification Experiments

‘Change new vs Time for Twa TCP Flows t= 42m)

{Change in cond vs Time for Twa TCP Flows = 162m)

(Change it cond ¥ Time for Twa TCP Flows = 324m)

‘Change in eond vs: Time for Two BIC Flows = 42m)

‘Change in omnd vs: Time for Twa BIC lows = 162 ms

‘Change in cond vs Time for Twa BIC Flows = 334m) —

‘Change in cond ¥ Tame for Twa CTCE Pls t= 2 ms)

‘Change in eord ¥: Tune for Twa CICP Flos = 162 ms)

‘Change neo vs: Thnefor Twa CTCP Flos (t= 324 ms)

‘Change in cond vs Time for Twa FAST Flows (toning enabled

‘Change in cvnd vs Tame for Twa FAST Flows tuning enabled

‘Change neon vs: Tine for Two FAST Flows (e-tning enabled = 324m)

‘Change neond Time for Two FAST Flows a= 80,742 m6)

‘Change inom vs Time for Twa FAST Flows (a= 80, = 162 me)

‘Change in cond Time for Two FAST Flows a= 80, r= 324 ns)

‘Change in cw vs Time for Twa FAST Flows a = 2007 = 42m)

‘Change cond vs, Tie for Two FAST Flows 163m)

‘Change in crn Te for Twa FAST Flows

‘Change in cond vs: Time for Two HSTCP Flows

‘Change cmd

‘Change in cond 3: Time for Twa H-TCP Flaws

‘Change in eond ys: Time for Twa H-TCP Flaws t= 1 ms ‘Change in cond vs Time for Tra IETP Fas t= 324 m9)

‘Change in cond vs Tame for Twa Sealable TCP Flos t= 42m)

‘Change in cond Tame for Twa Sealable TCP los at = 162 ms)

‘Change in cond Tae for Twa Sealable TCP Flaws t= 324s)

“Topology Adopted for Experiments

Secnarie Adopted fr Each Sula Condon

Period One

‘Cluster Atulss or 32 Conditions Using Data rom Tine Period One

‘Conditions Ordered from Least to Most Congested vs Retransmission Rat

(Chste Anas for Time Period One

‘Sample Plot Anarsng the Iftsence of Condition and Congestion Control Algol

‘Summary of StattallySigniieant Oullies a Tine Period One

Filtered Summary Plat for Tine Perlod One Identifying Stately Siac

Outirs wth Asocated Reta Efet > 10% Average Pero Goodpat on DD Flows fr Seven Congestion Control Algorithms

under Candin 4 ver od

Number of Aetve DD Flows for Seven Congston Control Algorthns under

Condit 4 over Three Time Periods

‘ares Packet Delivery Rate DD Hows for Seven

‘under Condition «over These Time Periods

66 lót

ns

204

206

a a0

6-15 Anayzing the Intacnce of Con

[Average Goodpul fr DD Flows 39) during Tame Peri Two au

616 naling the Inence of Condition and Congestion Conte Algorthn on

Congestion Window Sie during Tine Period Three a

617 Cluster Analysis Using Data from Time Period One ~ Algorithm 3 Excluded as

19 Chstering for Tne Period One ~ Annotated to Mentfy Dtintve Aritiw3 218 {620 Clustering for Time Period One ~ Algorithm 3 Omitted 26

621 Condon: Respnee Summa for Time Period One nie (622 Filtered Summary Pot for Tine Period One lenis Sati

‘Outirs wth Asblated Relative Eifect> 10° ar

628 Detaled Anal for Congestion Window Increase Rate in Tie Period One 3H {624 Detaled Analyt for Faw Completion Rate in Time Period One 3H {628 Detaled Analyt for Retransmission Rate in Time Period One 19 {626 Detaled Analy for NS Flow Completion Rate nTime Period One 219,

628 Clustering for Tine Period Two ~ Annoated io Ideal) Distinctive Aloe 2

629 Clstring for Tne Period Two ~ Algorithm 3 Omited 3

630 Condition Response Summary for Time Period Two m

631 Filtered Summary Plt for Time Peri Two Iden ing Stall Sigiicant

‘Outer wth Asblated Relate Eiect > 30° a

632 Detaled Analyt for Congestion Window Increase Rate in Tine Period Tw Bs {33 Detaled Analyt for Flow Completion Rate in Tune Period Two 3

634 Detaled Analysis for Retransmission Rate in Time Period Two hs {635 Detaled Analyt for Average Goopat on DI Flows in Tine Period Two bs

637 Detaled Analab fr Average Number of Connecting Flows ia Time Peri Two 236

38 Detaled Anal for Average Goody fon Long-lived Flow L1 in Tine Period Two 227

39 Detaled Analsis for Average Goodput on Longlived Fow L2 in Time Period Two - 227

640 Clasering for Time Period Three ~ Annotated to deny Dstinctive Algorithm 8-228

641 Glastering for Time Peri Three ~Algorihan 3 Omid 2»

642 Condidn-Response Summary for Tine Period Three 2»

643 Filtered mmary Pot for Time Period Three Mentiying tase Significant

Outrs wth Asocated Reais Efet > 30" 20 6-44 Detailed Anata or Congetion Window Inrease Rate in Tine Period Three mm

546 Detaled Anal for Retransmbsion Rate in ine Perid Three 3à

47 Detaled Anata for Average Good on DF Flows i Tie Period Tee an

48 Detaled Analsis for Average Number of Active DF Fowsin Time Period Three 282 6-49 Detaled Analysis for Average Number of Connecting Flows n Tne Perind Three — 238, {650 Detaited Anata for Average Cangestion Window Sie Time Period Thee as

Si Detaled Analisis fr Average Good on

82 Clustering for Totals Annotated wo Ident

53 Closering fr Totals— Atgorithm 3 Omited

54 Conditon-Response Sammars for Totals

{655 Detaled Anata or Agarqate Number of Flows Compicted aver 3

659

>a Mills et al Special Publication 800-282 xy

‘Change i Congestion Wind under HCP for Long-lived Flaw 12

der Condon

‘Change in Congestion Window under Scalable TCP for Long-Lved Flow 2 during

‘S00 Measurement Intervals within 1P2 wader Condon 21

“Average Congestion Window Size of DD Flows during BIC, FAST, HSTCP, TCP, Sealable TCP and TCP Reno TP3 under Condition 2 Tor

[vecage Congestion Window Size af DD Flows ding TP ender Condition 12 Tor

toFalling Congestion Windoy CC CC ST Congested

‘Change In Flow Stats over Three Time Periods under Condition 3 for Standard TO

‘Chang in Flow States over Three Time Periods under Condition & for Standard T

{Coster Anas or Time Period One

Detaled Anata of Retransnion Rate i Tame Period One when cluding

Response for Algoritin 3

‘Clustering for Time Period One~ each splot Annotated to Kentify Distinctive

Algorithme 0

‘Condition Response Summa for Time Period One ~ 10% Fier Applied

Detaled Anafss for Congestion Window Inrease Rate Per Faw in The Period One Detaled Anal for Flow Completion Rat in Time Peri One

Detnited Anata for Retranemision Rate n Time Perad One

Detaled Anaad fr \> Flow Completion Rate in Tan Period One

Detaled Anaya for Nomiber of Connecting Flaws in Tine Period One

Detaled Analss for Average Packets Outpt Per 200 ms Interval Time Period

One

Detailed Anal for Average Goodpat on Lonplived Fow 1.2 nTime Period One Detaled Anas for Average Bulfer Uizatin at Router Kain Tie Priod One — Custer Analyte for Tine Period Two —cach sub-plot Annotated to Idetiy

Distinctive Algorithms 9

‘Condition: Response Somme for Tine Period Twa

Detaled Anatase Congestion Window Inrease Rate n Tine Period Two

Detaled Anas for Packet Output Rate in Time Period Two

Detaled Anas for Flow Completion Rat in Tine Perad Two

Detailed Anas for Retransmsion Rate in Tine Perid Two

Detaited Anata for Averate Gondpat on IS Flows in Tine Period Two

Detaled Anata for Average Goodput on in Tne Period Two

Detaled Analss for Average Number of Connecting Flows during Time Period Two Detailed Anata for Bffer Uiation at Rover Ma daring Tne Period Tw

Clster Analss for Time Period These each sub-plot Anotated to Kdentty

Condition Response Summary for Tae Period Thee

‘Condition Response Summary for Tne Ptiod Tee 30% Filter Applied

Detaled nals of Congestion Window Inceise Rat for Tine Perod Te

Detailed Analysis of Packet Outpt Rate for Time Period Three

Detaled Analsb of Flo Completion Rate fr Tine Perid Three

Detaled Analysis of Retransntsion Rate fer Tne Period Three

Detaled Anais of Average Goodput on Longved Flow Lt

Detaited Analyst f Baer Uration n Router COs during Tie Period Three

‘Cluster Analysis or Totals ech sub-plat Anotated ta Tey Distintne

Algorithms 33

Condition Response Summary for Tota

Detailed Anal for Number of Packets input daring minute Sear

Detaled Analab fr Number of Flaws Completed over 2Sminute Scenario

Detaled Anas for Average SYN Rate for Connceting Flows ever 28-minute

Scenario

Detnited Analisis for Average Gondpat on Completed DD Haws over 75-minute

‘Change la Congestion Windovr under FAST-AT for Long-lived Flaw 12 daring 500 Measurement Intervat within TP2 under Condition 3

‘Change n Congestion Window under

MenssrementIntervat within TP2 under Condition 2 TGĐBÖT, ỬT Lived Flow Lt under Condon ©

‘Gondput ron (450 tụ 1800 for each Congestion Contra Algorithm on Lang

ined Flow L2 under Condon 8

‘Good rom r=4500 to 6500 for cach Congestion Control ined low L2 under Condition 8

‘Gondput rm e150 to 7500 for each Congestion Coral Algorit

ined Flow Li under Condition TẢ

‘Goodpat rm 7=450 to 7500 for sch Congestion Contra Lived Flow Ll ander Condon 26

Lived Flow Ll under Condition 42

‘Gondput rm e150 to 7500 for each Congestion Control Algorithm om Lang

ined Flow Ll under Condition 21

Average Congestion Window Size of DD Flows during TP3 onder Condition ¥for

BIC, FAST, FAST-AT, HSTCP, ICP, Sealable TCP and TCP Reno

“Average Congestion Window Size of DD Flos daring TP under Co

Average Goodput fr Flows Using Alternate Congestion Control Algorithy and

Flows Using TCP when Transferring Movies ona Very Fas Path with a Fast,

Interace Speed Given Lav nial low Start Thresold

att Most Congested (Law bi

ss {Components Analysis of Goodpats given High low-Start Threshold 33

59 hstraon of Biplo of PCT vs PC2 and Related Clustering a» S10 Seater Pot of 16100 ys 32(4V100 for Movies Transfer

‘wth Fas nteface Spesd Given a High Ina Shw-Start Thresholds FAST Allerate

Congestion Control Algorithm “0 Bar Graph for Movies Transfers overa Very Fast Path with Fast Interface Speed

‘gen igh InalSow-Start Thresbald during Condition 2 (Mo Congested) 340 Rank Matric for Algoritm 7 Seaable TC ita Slow Start Theesboll MỸ [Average Namber of Active Fos ander High Inia Sow: Start Threshold se

‘Average Number of Connecting ‘erage Rate of How Campletin ender igh Init Saw-Start Threshold ows under High ntl low-Start Threshold >4, a Average Flow Hetrunsofoion Rat under High Inia Slow-Start Threshold MS

‘Average Saoothed Round-Trip Tine under Hi As

‘Agategate Flows Completed under High Inia Sh-Start Threshold Me

‘Web Ohjects x Proportion of Flows Completed under High Init

“Average Number of Active Flows under Low bi er

‘Average Number of Connecting Flows under Lt âm

‘Average Rate of Flow Completion under Low Inia Slw-Start Threshold 3

‘erage Flo Retranamieton Rate ander Low Intl Stow-StartTheebol 50

‘Average Smoothed Round-Trip Tine uadee Los Ini Slo -Sart Thee 30

‘Aggregate Flows Completed under Law lial low-Start Threshold 3s Web Objects 4s Proportion of Flows Completed under Low Inia Slow-Siart,

526 Average Fow Consston Window Size onder Low Initial Stow Start Threshold 382

529 Average Good on Movies (High Inia lo-Start Tresba) ss

530 Average Goodput on Service Packs High Initial Slow-Start Thread) 356

5.33 Principal Component Is, Principal Component? (Mig lial Show Start Thre rom vera xe wảt — SeferPtaFComlmutonTCPEn

‘ery Fast Paths wlth Fast Interfaces High Ini low-S a8

835 32ihar Graphs (one foreach Simulated Conditn) plating Good

Non TCP Fe fọc Movie Transferred om Very Fas Paths ith Fst Interfaces

th ToHal Sư Start Threshold 7

36 Szller Pot of Goadput on TCP Flows vi Now¥CP Flows for Serve Packs

‘Transferred on Very Fast Paths with Fas Interaces High Ina Slw-Start

‘oot on Web Objects (Law inal Siw Start Tres) sot

542 Principal Component 1s Feincpal Component for Average Goodpa Daa (Low

843 Setter Pot of Goodput on TCP Flows vs Now-TCP Flows or Movies Transferred on

‘erg Fast Paths with Fast Interfaces (Low Iii SlowStart Thresbol)| 366 S44 32ibarGraplis one foreach slated condition) plting Good on TCP lows vs

(Low Intl Stow Start Threshold) xơ

S445 Sclter Mot of Goodput on TCP Flows vs Nom-TCP Flows for Service Packs

Teansored on Very Fast Pah wih Fan aterces (ov nial Slow -Saet

8.46 Bar Graphs pling Goodpat on TCP Flows vi Now-¥CP Fows for Service Packs

‘Transered on Very Fast Paths with Fast Ineraces (Low Inia Siw-Star

8-47 Scaler Mot of Goodpat on TCP Flows vs Nom¥CP Fows for Documents

‘Teanserel on Ver) Fast Pals wilh Fas Ineeaces (Lew Inia Show lar

sp Rank Matrix — 2) “FAST-AT igh Initial Sow: Start Threshold) a

833 Rank Mates ~ 2) “STC (igh Ina Sow- Start Thresbol) a

lable TCP (igh Inal Slw-Start Threshold) — 37% S56 it Rank Matrix ~y16(0) BIC (High Inia Slow-Start Threshold) 316

xơ Rank Matrix 5 16(0)—CTCP (High Inia loweStart Thresba) - 376

và Hank Mates )16(0)-FAST (igh Init Shw Start Threshold) — 7

‘TCP Goodput Rank Mate, y16(u)~FAST-AT (Low Intl SiowStart Threshold) 388

‘TCP Goodpat Rank Matra y16(0)—HSTCP (Low Intl Siow-Start Threshold) 386

‘TCP Goodpat Rank Matrix 16(0)—HFTCP (Low Intl ShreStart Threshold) = Seale TCP (Low ial Slow Start 386

378 Average vs Standard Deviation ln Goadput Rank Low lala Slow Star Threshold) 399

41 Conditions Ordered Least t Most Congested under igh Intl Slow-Start ‘Thresold 9

33 —— Distibton of Flow Sates for Six Conditions with Toresing Congestion mm 9.3 Average Number of Conecting lows under High Ina low-Stat Threshold 402

944 Average Retransmision Rate ude High Inia Slow-Start Thresbold aes 91S —Averape Flow Completion Rate ander High InialSlow-Start Threshold ‘os

956 Agaeegate Flows Completed under High Inia Sto Start Threshold ‘os

947 Average Smoothed Round-Trip Tne wader High nial Sio-Sart Thre ác

98 Web Objects Proportion of Flows Completed under High TU

#9 Moviesas Proportion of lows Completed under High Tita Sw Star Threshold 406

— Mills et al Special Publication §00-282 xi

erage Fw Congestion Window Size wader Iligh Initial Slow-Start Threshold

‘Average Goodputs on Service Pack under Combinations of Path Class and Interface

Speed

‘Average Goodputs on Documents under Combinations of Path Clas and Tnerace

Speed Average Goodpats on Web Objects under Combinations of Path Clas and Taterfnce Speed

Principal Component Iv, Principal Component 2 rom Average Ge

Large, Fst Network and High Il Stow Start Threshold

‘Goodpat on TCP Flows yx Now-TCP Flows for Mavis on Very Fast Pats with Fast

Interfaces ina Large, Vast Network with High nal Si-Star Threshold

Bar Graphs plating Gaodput Diferenees on TCP Flows Now ICP Flows for

Movies Transferred on Very Fast Paths with Fast Interfaces na Large, Fast Network

‘with a High tal Stow Start Threshold

‘Goodpat on TCP Flows yx Non-TCP Flows for Movies on Fast Path with Fast, Interfaces ina Large, Fant Network th High nial Stow-SartTheesold

Bar Graphs plating GaadputDiercnces on TCP Flos, Now-TCP Flos for

Movies Transferred on Fast Paths with Fast lntrfacs nw Large, Fas Network with Tigh ntl Stow Start Threshold

‘Goodpat on TCP Flows yx Non-TCP Flows for Serie Packs om Fast Path wth Fant Interfaces ina Large, Yast Network with High atl Sto: Start Threbold

‘Bar Graphs plating Goodput Diferenees on TCP Flows Now-TCP Flows for

Service Packs Transferred on Fast Paths with Fas Interfaces na Largs ast

‘Network wth Ih ntl Stow-Sart Threshold

Rank Matrix = 2) IC (Large, Fast Network High iia Slow Start — Rank Mates ~ 2) CTP (Large, FAST (Large Fast Network, High Initial Fas Network, High nial Show tar) Slow Slar)

FAST-AT Wearge, Fast Neworky High Init Son

rp Duaina

“Average vs Standard Deviation in Goodpat Rank (Large, Fast Network,

Slow -Start Threshold)

igh Tata

Response curves of ro hypothetical TCP vartants TCPO and TCP! and hee

Response curve for CUBIC with a 250% eror region vs TCP Revo

Response carvesfor CUBIC and CTCP with corresponding errr exons

409 a0

an a2

a

a

ais ais a9

‘Changes in Throughput Falnes with Increasing Bulfr Se —

‘Changes in Average Throughput with Inereasing Buffer Sie

Main Etfcs Plots or Top Four Pingpal Components

{Y-V"X plat for Responses 7 and 922

Maint Plot for Response i (Average Number of Active Flow)

Main-Eets Plot for Response 10 (Average Retransmission Rate)

Main-Eets Pt for Response 11 (Average Congestion Window Sez)

Miễn Tu ot for Response 322 (Average Throughput on Sow)

‘Man Etfects Plot for Response 318 (Average Stoothed Round-Teip Tae)

Maln-fets Plat for Response 34 Average Packets Output per Measurement

Sample Multfactr Seater Pt

Sample Main Etects Plt

Sample Interaction fects Mate

Sample lock Pts

‘Simple Camlative Resid Standard Deviation Plt

Sample Contour Plt of Tw Dominant Factors

Mills et al Special Publication 800-282

su

26 a7

20

see

‘Sommary of Flow-Class Measures Reported by MesoNt Se

‘Semmiry of LongLived Flow Measores Reported by MesoNet—

Summary af Added Meares Reported by MewNet fr Aces Roulor bì

‘Characteriste Performance for MesoNet a Two Experiments “

‘S10 Pracesing Requirements for MesoNet in To Experiments — m SIL Memory Requirements for MesaNet in Twa Experiments 6

Responses Characterizing Instantaneous Throughput for Active lows hy Flow Cass) 79

nt and Purpose a 10 Pats the 1-Step Grapes! Analyst &% Simalaton-Contrl Parameters ™ ” fer Adapting Network Characieriies 3 ering Properties of Sources and Beceem 3 Parameters Conralling Source Startup Pattern Parameters Related to TCP Operation 3 93 Relations anong the Speed of Backbone Routers and Other Router Type 38 Relation between Factors and Number and Distrbuton of Sources m Relation betcen Factors and Number and Distribution of Recess m Relation between Factors and Dstibution of Flow Clases o7 Buffers for Combinations af Round-rip Propagation Delay 1) and Capaciy (98

‘Characters of Procesors Executing Suton Run » Execution Time Reuirvd for Each Simulation Run sút Responses Sletd for Investzaton in Sens 107

too

ho Definition of Major P Rank Analyse hand on ny

‘Srmbas and Definitions Used o Model BIC Congestion Avoidance Procedors tơ

‘Sas nd Deion Used o Mode CTCP Congestion Asolaee Procelarx.— — ID

59 Capacity ofthe Dumblell Topology wth Varions Round-Teip Times _ ny E11 Bandwidth Fairness Jun’ Indes for Sinulated Congestion Control Meshanisns THƠ

61 Congestion Control Mechanisms Compared Hà

62 Definition of Three Path Clases HH c3 —— RaBadmevFadocvSeertelturCompariapCommaonontulAishamams Hệ

6S Domain View of Router Speeds Hồ

Path Propagation Days Sinolated Hồ

umber of Simulated Sources ng Fed Simulation Control Parameters ng Fed Parameters Spetyng Simulated User Trae ned Parameters Specifying Long ved Flows 188 9

‘Characters of Compate Servers Used to Execute the Simulations Pracesing Requirement or Simlations Mapped to Specie Compute Servers 196 196 Format Adopted for Fach Tine-Perad Data Fle 193 Format Adopted for Reporting Agarezate Measures I9 Flows Completed per 200 ms interval and Total Completions Tor DD Flows a Tine

Period Two under Condition 4 a

632 Aveenge Minin ond Maxam Goodpal en DD Flog for Each Conasaton

Control Algorithm during TP2 when Averaged over AIL 32 Conditions 20

TA Cangeston Control Mechanisms Compared ga

T-Ạ Fixed Parameters Related to Sources nd Receivers — 255

34 nntanated Robustness Conditions hà

73 Domain View of Router Speeds 286

Algorithm in a Large, Fast Network and a Seaed-Down Network 2

79 Comparing Number af Sinasted Flows and Packets fora Large, Fast Network and

Sealed-Down Network under Al Congestion Control Algorithms ga

‘Goodpats on DD Flows Averaged overall 2 Condition for Each Time Period Per Flow Goodputs for Lange wed Fw Lt Averaged over ll 3 Condi for a

Per Flos Gaodputs for Lang-Lived Flow 12 Averaged over ll 3 Condilons for

Mills et al Special Publication §00-282 xe

73 Per Flow Goodputs for LangsLived Hw 3 Averaged over 2 Conditions for

TIS Time unl Long-ined Flows Recover Maximum Transfer Rate in TPS fr

716 Average Goodpu or Fach Congestion Control Alot on Three Longe

Flows daring 12 under Condition san

7417 Time ual LongeLved Flow L1 Reaches Masia Transfer Kale a WPI for Three

7-18 Time until LongLined Flow LI Recovers Mama Transfr Rae in TPS or Thre

under Each af Three Uncongested Conditions 4

“1⁄40 ‘Average Goodput on LongLived Flow Li for Each Congestion Control Algorithm

ach ofthe Three Time Periods under Most Congested Condltion 2 us 5:1 Alternate Congestion Control Regimes Compared 316

2 Redwine actar Adopled for Compuring Congestion Contr Neckar mm

#3 Prahabity Distbutins fr Fle of Various Stes 3

34 Fed Parameters for Sring Files 3i8

55 Four Dimensions Defining Flo Groape 3

#7 Computing Target Minimums for Document Transfers with Combinations of

59 Fed Souree and Receiver Parameters 33 5-10 Proportion of Soures and Receivers Placed under Specie Rowtr Cis Bs KHI Prubability of Fons Transting Sposfic Path Clases mà

$12 Fld Simulation Control Parameters tế B13 Two-Factor2™ Orthogonal Fractional Factorial Design Template ms S14 The 32 Simulated Cantons used to compare Each Combination of Congestion Conirt Algeritu and Inia Stow Start Threstld 26

317 Simulated Propagation Delays a

18 Characterization of Sole fer Ses Rr S19 Measured Responses Charateriaing Macroscopic Network Boban ior m

520 Measured Response Characterizing User Experience for Each Flo Group a

S21 Comparing Resource Requirements or Simulating One Hour of Network Operation

‘under 32 Conditions with High and Low ial low-Start Thresholds

#33 Comparing Flows Completed and Data Packets Sent when Simlating One Hour of [Network Operation under 32 Conditions with High and Low Ini Stow Start,

Data Format Summarizing Responses 1 fo y16Tor AM Algorithms and Conditions 36 Data Format Summarizing User Experience for One Flow Group as Data Format Summarizing User Experience fr One Fl Groep under A

1826 Average Goodpt per Faw Group under Each Alternate Congestion Control

‘Algorithm (High Initial Slow-Start Threshold) 383

827 _Aserage Goodput pr Fo Group on TCP Flaws Competing with Each Arma

‘Congestion Control Algorithm (ih Initial Slow-Start Threshold 3s

828 Average Goodput per Flow Group under Each Alternate Congestion Contra

#39 _Arerage Goodput pr Fw Group on TCP Flows Competing with Each Akernate

‘Congestion Control Algorithm (Low Initial Slow-Start Thresho)

5.30 Range of Goodpat Differences (4) for Flow Groups unde High ad Low Init

32 Summary Average and Standard Deviation in Goodpat und TCP Goodpat Rank for

AMI Congestion Control Algorithms Low til Slow Start Threshold) a0

91 Comparten of periment with Congestion Conte Alors in Small Network ys Experiment in Large Fast Never ws 9.2 Rabusines Factors Adopted for Comparing Congestion Control Mechanisms tì

#3 —— Kerisetfacrs Adnpledfar ComparingCowgeviowComtml Meehanteme 86

94 ‘ToorLevel 2 Orthogonal Fractional Factorial Design 7

59 Characteriation of Simulated BalferSes 9g

99 Comparing Nomber of Simulated Flows and Packets Tora Small Network anda

9:10 Average Goodput per Flow Group under Fach Ateraate Congestion Contol

[Aiur fora Large, Fan Network th Igh ntl Show Start Threshold _ ‘07

91 Average Goodpt pee Flow Group on TCP Flaws Competing wih Each Altrnale

‘Algor for a Large, Fast Network wth High Intl Show Start Threshold os

9412 Range “Theesold for Smal Show Network od for of Goodput Differences (%) for Flow Groups under High Initial Slow Start a6

Alternate Congestion Control Alorthns and for Competing TCP Fas (Large ast

‘Network High Inia Slow Start Threshold) as

1041 Comparing Four Characters of Individual Alternate Congestion Control ‘iors - 46

AcL_Bstimatd throughpat for CUBIC i plas for 1000 concurrent Rows om ink with a

123 ps eapact (for 1 KB packets) 0

‘A Estimated throushpot for CTCP in pms for 1000 concurrent flows on aink with w 122 pms enpacty (tr TRE packets) 0

AS Estimated throughput for TCP Reno ln pis for 100 concurrent Howson aia with

1122 pins capaci for 1 KB packets) 40

Bl One-Way Propagtion Delay on Fach Link inthe Simulated Topology 485

12 Character of Three Flow Sel Simlatd inthe Experiment he

RS MosNet Parameter Stings forthe Experiment ác H4 DamlnVievoftheSimulaedNgvadtCharaetuhdls mm

BS Configuration of Compute Server fr Simulations a8

BG Resource Regurements fr Simulations a8

Mills et al Special Publication 800-282 xin,

C57 Changes in Ranking Atrtutable to Each Factor 39

Dl HdenHt and Purpose of 10 Pats the 1Step Graphical Anabsi =

Table of Acronyms

‘Acronym Definition

ACK ‘Acknowledgment

AvfSWO ‘Average Fle Size for Web Objects

AdhT ‘average Think Time

AWND Advertised Window

sic Binary Increase Congestion contol

Rs, Backbone Router Speed

c ‘capacity

a Celluis Avtomata

các Correlation Analysis and Clustering

ou Central Processing Unit

cho Curent Retransmission Time Out

cep ‘Compound TcP

¬ Congestion Window

es Discrete-vent Simulation

or Data segment or packet)

pwn Delay Window (used by C1CP)

rast Fast Ative Queue Management Scalable TCP

FASLAT Fast Aetive Queue Management Scalable TCP with Alpha Tuning

oe Gigabytes (Giga denotes bilion)

‘Sops lg2bi per second (siga denotes billion)

GENI Global Enviconment for Network Innovation

He Giga Hertz (Giga denotesbilion)

ste High Speed T¢P

HICP HamitonTCP

rene Internet Congestion Control Research Group

e InternetProtocol

tr Internet Research Task Force

Ep Internet Serie Provider

tr Information Technology Laboratory

ist National institute of Standards and Technology

st National Science Foundation

0£ OelbogonalFradionalFacuial

Pe Peerto-Peet

Mills et al Special Publication §00-282 xi

packets per tie step packets per secand packets per milisecond Probability ofa Fast Host Probability ofa Larger File

‘Queue Sizing Algorithm Receiver Distribution Retransmission Time Out Round Trip Time

Receiver Window Standard Deviation Source Ditrbution Sealing For Sources and Receivers Simulation Language with extensibility Square Root

Square Root Smoothed Round Trip Time slow star threshold Scalable TCP

‘Transmission Control Protocol User oatagram Protocol World-wide Web

1 Introduction

Society is becoming increasingly reliant on large networked information systems for eommeree, communication, education, enterainment and government [Despite] society's profound dependence on neoworks, fundamental knowledge about them is primitive [Global] communication networks have quite advanced technological implementations but their behavior under sess still cannot be predicted reliably There

is no science today that offers the fundamental knowledge necessary 10 design larg complex networks [so] that their behaviors can be predieted prior 10 building them [104) This lck of knowledge grows more acute as society moves toward service-oriented architectures (102-103) that deploy software, platfonms and infrastructure as distributed services accessible through networks,

‘Why are large distibuted systems so dificult to prediet? Such systems exhibit tlobal behavior that arises from independent decisions made by many’ simultaneous

‘ctors, which adapt their behavior based on local measurements of system stale, AS a result of actor adaptations, global system behavior may change, influencing subsequent measurements, and leading to further adaptations This continuous eycle of measurement and adaptation produces a time-varying global behavior that drives the performance

‘experienced by individual actors within spatiotemporal regions of a large distributed system Thus, (0 truly understand and predict behaviors in such systems requires techniques to model and analyze designs at large seale, Such techniques are currently beyond the tate ofthe art, as practiced by network researchers

[As part of a team of researchers [105] at NIST, we are investigating methods to

‘model and analyze distibuted systems, such as the Intemet, computational grids, service oriented architectures and computing clouds As part of this investigation, the study reported here develops, applies and evaluates a coherent set of modeling and analysis methods for distributed information systems of large spatiotemporal scale The methods are adapted from techniques often applied by NIST scientists to study physical systems

In this study, we develop methods to investigate global system behavior within the context of challenge problem: comparing some proposed changes to the standard congestion control algorithm [9-10] for the Intemet Congestion control procedures are implemented as par of the transmission control protocol (TCP) that operates within every computer attached to the global Intermet, Numerous researchers [46-51] have forecast changes in relationships among bandwidth and propagation delay asthe speed of network links increases, These researchers predict that the current version of TCP will prove inadequate, leading to substantial underutilization in-network resources and preventing fend users from achieving high tansfer rales, Such predictions have stimulated researchers lo propose allemale congestion contol algorithms [52-61] intended 1 achieve higher’ network utilization and better user performance Evaluating the

‘implications of adopting proposed changes 10 TCP congestion control procedures requires investigating plobal behaviors that result when such changes are deployed on a large scale throughout an Intemet-like network, The curent study provides such an investigation

We hegin (in Chapter 2) with a discussion of the challenge problem and the current state ofthe art with respect to investigating proposed Internet congestion contol algorithms We outline various approaches that we considered for modeling and analysis

and we deseribe the approach we selected We introduce Five hard problems we needed to Solve in order to implement our approach and we discuss the solutions we adopted In Chapter 3, we describe MesoNet, a medium scale, diserete-event simulation model that

we created for use in this study MesoNet allowed us to expose candidate congestion control algorithms to a wide variety of network conditions, We subjected MesoNet 10 sensitivity analyses, as documented in Chapter 4 and in Appendix C These sensitivity analyses helped us to gain confidence that MesoNet provides a suitable model for TCP networks, and also enabled us to identify the most important parameters influencing

‘MesoNet behavior As part of our sensitivity analyses, we employ a NIST-developed 10- step graphical analysis process, which is descriped in Appendix D In Chapter 5 we explain our models for various eongestion contol algorithms and we document key

‘empirical comparisons used to verify model correctness The bulk of the study consists of six experiments, which we describe in Chapters 6 through 9 As we discuss in Chapter 2, these experiments were not constructed as an integral campaign, but rather arose through

1 process of iterative refinement, where findings from previous experiments suggested

‘useful directions for subsequent experiments We frst compare (Chapter 6) congestion control regimes in a lange, fast network simulation and then repeat the comparison (Chapter 7) in a simulated network with smaller size and slower speeds In Chapter 8, we enlarge the trafic classes considered, while comparing the congestion control algorithms ina network where some flows use standard TCP and some use alternate algorithms In Chapter 9, we repeat an experiment from Chapter 8 but in a larger, faster simulated network, where theorists suggest alternate congestion control algorthims could provide best advantage Taken together, these experiments compare the behavior of seven congestion control algorithms under a wide range of simulated conditions We generate sufficient information to draw some conclusions in Chapter 10 about the congestion control algorthnss Chapter ID also provides an evaluation of the methods that we

‘developed and applied We include some appendices to document auxiliary investigation

of analytical (Appendix A) and hybrid (Appendix B) models of TCP networks

‘This study may interest two different audiences: (1) those seeking to understand and evaluate methods to model and analyze behavior in large, distibuted information systems and (2) those aiming to compare proposed changes in algorithms for te Intemet Readers in the fist audience can expect (0 learn about various modeling, experiment design and statistical analysis techniques applied to study dynamies in complex systems

In addition, such readers may benefit from our Findings with separd tothe strengths and weaknesses of the techniques we applied Readers in the second audience can expect (0 eam how to model a data communications network with a manageable set of parameters, Inaddition, such readers may benefit from learning how we Tet measurement data (rather than preconceived metrics) drive our comparison of alternative congestion conol algorithms Mindful of these two different audiences, we attempt to provide a sufficient level of explanation to engage every reader We explain our modeling and analysis

‘methods in detail so that networking experts can follow our methods, And we provide sufficient torial information to allow those readers who are nat networking experts to follow our challenge problem and related findings Where appropriate, we also provide references additional sources where readers in each audience can pursue more information

Mills et al Special Publication 800-282 3

We ean summarize the contributions of this study along several lines First, we define and demonstate a coherent set of modeling and analysis methods that can be used

‘o investigate behavior in distributed systems of large spatiotemporal scale The methods

‘we develop represent an advance in the state of the art, as curently practiced by network researchers Second, we evaluate our modeling and analysis methods inthe context of a challenge problem that investigates behavior of various proposed Intemet congestion contol algorithms The challenge problem is of current inlewest to industrial and academic researchers within the Intemet Congestion Control Research Group (ICCRG)

of the Invemet Research Task Force (IRTF) Thitd, we provide conclusions and recommendations with respect to the congestion contr algorithms that we study We demonstrate that our methods lead to insights that have not been obtained using existing methods Fourth, we describe a medium-scale, disrete-event network simulator that we

«developed for our study The simulator, called MesoNet, can be efficiently parameterized and allows feasible simulation of high-speed networks transporting hundreds of thousands of simultaneous flows, The most commonly used network simulators are incapable of supporting such large-scale models, Fifth, we suggest an approach that right improve the accuracy of exiting analytical models for Internet congestion contol algorithms We anticipate future work to inchude improved analytical models within existing flui-flow simulation frameworks in an effort t obtain accurate predictions regarding spatiotemporal bchavior in large networks

‘=p

“DMG:

oe Ee SUDNE|MUUISE,

Eas yuna

gq ll"=

manh

= 0B]

2 Method and Related Work

The work deseribed inthis report supports an overarching goal to develop and evaluate a coherent set of methods that can be applied to understand behavior in large distributed systems, such as the Intemet, computational grids, service-oriented architectures and computing clouds Large distributed systems may exhibit emergent behaviors, which are slobal behaviors arising from independent decisions made by many simultaneous actors,

‘which adapt their behavior based on local measurements of system state, AS a result of actor adaptations, system state shifts, influencing subsequent measurements made by the actors, which leads to further adaptations This continuous cycle of measurement, adaptation and changing system state produces a time-varying (emergent) global behavior that influences performance experienced by individual actors within speci spatiotemporal regions of a large disiibuted system For this reason, any proposed changes in decision algorithms taken by actors must be examined within the context of large spatiotemporal seale in order to predict the effets of such algorithms on overall, system behavior, as well asthe resulting implication for individual actors

In this study, we develop methods to investigate global system behavior within the context ofa chalienge problem: comparing selected proposed changes tothe standard congestion control algorithm [9-10] for the Internet AS we show later, in Chapter 10, using our methods we were able to draw conclusions (1) about likely network-wide behaviors and user experiences that may arise if the Intemet adopts any one of the algorithms we studied and (2) about te eifcaey ofthe methods we used In this chapter,

\we introduce the challenge prablem, describe the current state-of-the-art techniques used

‘0 address the problem and outline a proposed advance inthe state of the art We consider some approaches that might be adopted to achieve our intended improvement in practice and then we explain the approach we adopted forthe curent study We identity five hard problems we had to solve to develop our approach and we discuss some possible solutions to the problems and centify the solutions we adopted for the curent study We conclude with an argument that the methods we develop and apply in the curent study should be generally applicable toa wide array of large distributed systems

2.1 Challenge Problem

The fundamental design of the Internet protocol suite [3] assumes that network element, such as routers, are relatively simple ~ receiving, buffering and forwarding packets among connected inks and dropping packets when buffers are insufficient to accommodate arriving packets Under this assumption, computers connected to the Intemet must implement decision algorithms to pace the rate at which packets are injected into the network Such decision algorithms, known typically as congestion control mechanisms, operate independently for each network flow between a source and receiver The overall network, wih a goal of achieving satisfaciory service and a fa distribution of resources among all simultaneously active flows, relies upon each network source to measure congestion and dhen adapt the rate at which the source injets packets ino the network — injecting faster when congestion is low and slower when congestion is high Thus, congestion in the Intemet is an emergent property of the simultaneous

‘operation of many independent sources