A framework for formalization and characterization of simulation performance 5

We have proposed a time independent measure called strictness for quantifying the degree of event dependency of a simulation event ordering,... Figure 5.1: Framework for Formalization an

Chapter 5

Conclusions

5.1 Summary

This thesis has proposed a framework for the formalization and characterization of

simulation performance The proposed framework provides a basis for understanding

and analyzing simulation performance from the simulation problem to the simulation

implementation

Figure 5.1 summarizes the proposed framework We formalize simulation event

orderings that provide the theoretical foundation for simulation performance analysis

based on the partially ordered set (poset) We characterize simulation performance in

three layers: physical system, simulation model and simulator Our analysis focuses on

time (event parallelism) and space (memory requirement) performance Because of the

different definitions of time used at different layers, a normalization process is required

to allow time performance comparison across layers Event parallelism depends on

event dependency and time We have proposed a time independent measure called

strictness for quantifying the degree of event dependency of a simulation event ordering,

and a relation called stricter to compare the degree of event dependency of simulation

event orderings

Figure 5.1: Framework for Formalization and

Characterization of Simulation Performance

In summary, this thesis offers three main contributions: formalization of event orderings,

time and space performance characterization, and strictness measurement

5.1.1 Formalization of Simulation Event Orderings

We have proposed the formalization of simulation event orderings based on poset Poset

is a branch of discrete mathematics that studies the ordering of elements in a set In event ordering, we study the ordering of events in a given set of events At the physical system, events are ordered based on their physical time of occurrence A number of event orderings can be used at the simulation model layer A simulator maintains a specific

Physical System

Simulation Model

Simulator

Event

Ordering

Perf

Perf

Perf

Π prob

Mprob

Π ord

Mord

Π sync

Msync, Mtot, Mshr, Mdst

Strictness (ς)

Stricter (< ς )

Poset

Perf = performance measure

maintained by a number of simulators (sequential and parallel) This formalization provides a theoretical foundation for carrying out performance analysis of simulation The existing theories in poset such as Dilworth’s chain covering theorem can be applied

to explain the performance of different simulation event orderings The proposed formalization can also be extended to other computer systems that involve ordering in their operations such as memory operation orderings in the memory consistency model [CULL99], instruction orderings in processor design [HENN03], and job orderings in grid computing [THAI03]

5.1.2 Time and Space Performance Characterization

We have proposed characterizing simulation performance along the three natural

boundaries in simulation, i.e., physical system, simulation model and simulator

(implementation) This thesis focuses on the simulation of physical systems that can be modeled as queuing networks Therefore, at the physical system layer, we view a physical system as a queuing network At the simulation model layer, based on the virtual time paradigm [JEFF85], we view a simulation model as a set of logical processes (LPs) Every LP models a physical process (service center) in the physical system The interaction among physical processes in the physical system is modeled by exchanging events among LPs in the simulation model At the simulator layer, a simulator that implements a simulation model is executed on a platform consisting of one or more physical processors (PPs)

We have formalized event parallelism and memory requirement at each of the layers Event parallelism is defined as the number of events executed per unit of time At the

physical system layer, the physical time unit is used At the simulation model layer, the time unit is timestep At the simulator layer, the wall-clock time unit is used Analysis at each layer can be done independently To compare event parallelism at different layers, normalization is necessary because the time units used at the three layers differ

The memory requirement at the physical system layer is derived from the maximum queue size of each service center in the system At the simulation model layer, the memory requirement is derived from the maximum event list size of each LP At the simulator layer, the memory requirement is derived from the maximum size of the data structure that is used for synchronization purposes The memory requirement at each layer can be analyzed independently

The total memory requirement is measured at the simulator layer At runtime, in addition

to the data structure that is used for synchronization purposes, other data structures (such

as a linked list) are also used to implement queues and event lists These data structures constitute the total memory requirement of simulation

The proposed three-layer performance characterization can be extended to other areas In shared-memory computer architecture, for example, a program that is going to be

executed constitutes the physical system layer, a memory consistency model that

specifies the ordering of memory operations independent of implementation constitutes

the model layer, and an implementation of the memory consistency model constitutes the

implementation layer With this characterization, we can analyze the performance from

the physical system layer to the implementation layer Similarly, in processor design (to exploit instruction level parallelism), a program that is going to be executed constitutes

the physical system layer, the different instruction orderings analyzed using an idealized processor model constitutes the model layer, and an implementation of the instruction orderings on a real processor constitutes the implementation layer

5.1.3 Strictness of Event Orderings

We have proposed a relation called stricter and a measure called strictness to compare

and quantify the degree of event dependency of event orderings respectively An event

order R 1 is stricter than R 2 if event x is ordered before event y in R 2 then x is also ordered

The strictness of an event ordering is measured from the number of comparable event pairs which is divided by the number of comparable event pairs in the total event

ordering A strictness value of x means that the probability of two events being comparable is x If all events are comparable (non comparable) in an event ordering, then

the strictness of the event ordering is one (zero)

The relation stricter has been used in the memory consistency model [GHAR95] It can

be extended to compare the degree of dependency of different orderings in areas such as

instruction orderings in the processor design [HENN03] The measure strictness can

also be extended to measure the degree of dependency in an ordering such as memory operation orderings in the memory consistency model [CULL99] and instruction orderings in processor design [HENN03]

5.2 Future Works

We highlight three possible research areas that can be further explored:

a Hardware Implementation

The formalization of event orderings is motivated by research in the memory consistency model A number of hardware implementations have been built based on the existing memory consistency models [CULL99, GHAR95] Learning from research in the memory consistency model, it may be possible to explore the hardware implementation of a simulation event ordering This may improve simulation performance

b Empirical Study on Other Protocols

This thesis has used only a CMB protocol with demand driven optimization in the experiment We have formalized a number of event orders and each event order can

be implemented in many ways, resulting in different simulation protocols Therefore, empirical study of the various protocols will be useful in supporting the proposed framework Further, the proposed framework also provides the foundation for studying future event orderings and new protocol implementations

c Extreme Value Analysis

We have formalized the memory requirement based on the total of a maximum value, such as maximum queue size at the physical system layer, maximum event list size at the simulation model layer, and maximum buffer size at the simulator layer The stochastic behavior of a maximum value is the focus of research in the extreme value theory [COLE01] Therefore, the extreme value theory may provide some analytic model that can explain the stochastic behavior of the memory requirement