Standardized Functional Verification- P6 pdf

Relations among variables for a target of verification Choose values for these variables: To establish: Internal connectivity Instance External connectivity Context Power, clocking, and

<’

struct gen_crc {

crc: int;

crc = compute_crc(data);

keep impose_error == TRUE => crc == crc+1;

// Adding 1 to computed value yields a crc error }

‘>

2.13 Generating Excitement

Choosing the values of the external conditional variables and the values of

stimulus variables determines the excitation of an instance within its

con-text

Pseudo-random excitation is currently the province of commercially available pseudo-random test generators

2.14 Special Cases

The foregoing categories of variables (connectivity, activation, conditions, and stimuli and response) form the basis for pseudo-randomly exercising the target However, certain aspects of a target’s functionality may not be readily exercised in a pseudo-random manner, but instead require special treatment

These are best defined as special cases, and are often exercised with

deterministic tests, or with deterministic test sequences within a pseudo-random sequence

Most special cases will be associated with stimuli or responses, but there may also be other things to verify that just don’t seem to fit neatly into any subset This should not impede progress in interpreting the design specifi-cations – just write them down as for later classification Eventually, with time and increasing familiarity with the design, special cases in this latter category will be recast as variables of suitable type with their respective ranges Treating them as special cases early in the project is a practical convenience that enables progress in spite of a few “loose ends”

Interpreting functionality related to reset often accumulates a number of special cases For example, checking the state of the target immediately af-ter de-assertion of reset (whether hard reset or soft reset) is more readily accomplished with a deterministic test and merits description as one or

36 Chapter 2 – Analytical Foundation

more special cases Similarly, checking for state preservation upon receiv-ing a soft reset constitutes another batch of special cases On the other hand, these are better regarded as variables of response whose values are dependent on the type of reset

If indirect conditions are defined for a target, it is important that the dy-namic transition from one value to another during the application of

stim-uli under given conditions be recorded as special cases to be exercised and observed

Other external activity such as the connection or disconnection of a ca-ble, entering or leaving power-save states, and entering or leaving test modes (morphing the target) can also be recorded as special cases during early stages of a project

Each of the special cases discussed so far fall into the category

fre-quently referred to as scenarios As will be seen later, these scenarios

ac-tually correspond to transitions among values of standard variables, but for the time being they will simply remain on the list as special cases

The second category of special cases lists properties 5 of the target

These describe higher-level (or transcendental) behaviors of the target that

are usually not verifiable via CRV, such as the performance (in MIPS) of a processor Verifying performance is likely to need additional testing above and beyond CRV, which tends to focuses on correctness and completeness rather than speediness Properties often encompass behavior of multiple clock-domains and are often expressed with inexact requirements

Passing a compliance test (such as for USB or PCI-Express) is also a property of the target.6 Other properties may be based on aesthetic evalua-tions, such as the quality of an image encoded and decoded by the target,

or the sound fidelity produced by the target Cybernetic properties such as responsiveness to human interaction (in controlling a commercial aircraft, for example) would be listed here as well

5 Note that the term properties is also used in the context of model checking where

a given property of the RTL is to be checked Properties listed as special cases are not based on RTL, but are based only on the requirements stated in the de-sign specifications

6 Passing a compliance test does not mean that the risk of a functional bug is zero Examples abound of products (including IP) that pass compliance tests but still have bugs

2.14.1 Example – Special Case

A classic example of a property as a special case comes to us from the specifications for Ethernet (IEEE 802.3) that states requirements on the distribution of the back-off slot times

Ethernet uses a collision-detect multiple-access algorithm on a shared medium, the wire carrying the electrical signals When two (or more) de-vices happen to drive the wire simultaneously, both (or all) dede-vices must stop driving the wire and wait some specified period of time until attempt-ing to drive the wire again The time each must wait is referred to as the back-off slot time, where a slot corresponds to a specific number of bit times

Testing for this requirement is achieved within a network of compliant, commercially available devices The back-off slot times used by the device being tested are recorded and then tested for a probabilistic uniform distri-bution

Verifying this particular requirement doesn’t lend itself to CRV testing and isn’t readily verified in simulation Similarly, functional requirements related to liveness, fairness, or performance of a target may require treat-ment as a special case outside the context of CRV

2.15 Summary

The taxonomy defined in this chapter provides a consistent and compre-hensive nomenclature for all aspects of functional verification regardless

of the size or complexity of the digital system to be verified The entire function space can be characterized completely and exhaustively by a well-defined set of standard variables using the linear algebra for digital systems The premise stated at the beginning of this chapter bears repeat-ing here:

From the standpoint of functional verification, the

func-tionality of a digital system, including both the target of

verification as well as its surrounding environment, is

characterized completely by the variability inherent in the

definition of the system

Different teams will interpret specification documents to yield different standard variables, surely differing in name but also differing in definition,

38 Chapter 2 – Analytical Foundation

but the sets of variables will be equivalent, each forming orthogonal basis sets Any non-equivalence indicates one or “bugs” in a document because

it is not sufficiently clear or ambiguous in its interpretation

Table 2.1 summarizes this “loosely orthogonal system of basis variables.”

Table 2.1 Relations among variables for a target of verification

Choose values for these variables: To establish:

Internal connectivity Instance

External connectivity Context

Power, clocking, and reset Activation

Morph-related conditions and stimuli Morph

Internal conditions Operation

External conditions

Stimuli

The variables discussed in this chapter already exist in testbenches around the world However, they simply have not been partitioned into the standard framework And, without this standard partitioning, it is ex-tremely difficult to compare different verification projects or to use these projects to assess risk for future projects

With this set of standard variables it is now possible to determine the size of the functional space in terms of its points and arcs and, therefore, determine what is required to achieve functional closure, that is, exhaus-tive functional coverage This will be the topic of the next chapter

References

Bailey B, Martin G, and Anderson T (ed.) (2005) Taxonomies for the Develop-ment and Verification of Digital Systems Springer

Lachish O, Marcus E, Ur S, Ziv A (2002) Hole Analysis for Functional Coverage

Data, Proceedings of the 39th conference on Design Automation, ACM Press

Verisity Design Inc (2002) e Language Reference Manual www.ieee1647.org Verisity Design Inc (2005) Spec-Based Verification, whitepaper

The previous chapter described a method of interpretation of natural-language specification documents into a standard set of variables and their ranges that is applicable to all possible digital systems Each unique com-bination of values of these variables determines a single function point within the functional space These function points determine the functional space in a mathematically precise manner, allowing us to determine the size of this space and, consequently, determine what it means to achieve functional closure Value transition graphs provide a technique for visual-izing this space in a useful way

3.1 Functional Closure

Much recent discussion in the industry has focused on “functional closure” but without providing a solid, substantive definition for this highly desir-able outcome Intuition tells us that functional closure must exist because, after all, the functionality can be expressed with a finite number of transis-tors What does it really mean to achieve functional closure?

This term is borrowed from timing domain analysis where closure is a genuine outcome To reach timing closure is to adjust the placement of transistors and the routing of interconnecting wires and polysilicon such that each and every path propagates signals within a predetermined inter-val of time so that setup times at destinations are satisfied Software tools

make exhaustive lists of signal paths and then determine whether all paths

have no negative slack When a signal has positive slack, it means that the signal reaches its destination with time to spare, whether a few picosec-onds or many nanosecpicosec-onds A signal with negative slack reaches its desti-nation too late When all paths have no negative slack, nothing arrives late Some texts refer to “functional closure” or “coverage closure,” but while also claiming that the size of the space over which this “closure” is reached cannot be determined This is not correct, as we will see in the analysis in this chapter

A survey of current literature finds many technical papers and articles that tell us, “A good test plan should list all the interesting test cases to

40 Chapter 3 – Exploring Functional Space

verify the design.” However, there is no objective way to separate “inter-esting” from “uninter“inter-esting” test cases If so, there would be a software program that could accept as its input these “test cases” in some yet to be defined form and then decide algorithmically whether it is interesting or uninteresting Clearly, this approach does not seem promising

Similar remarks appear in the current literature regarding coverage, in-forming the reader to “decide on the interesting data fields/registers … im-portant state machines, … interesting interactions between some of the above states or data.” Again, we have no objective way to make these de-cisions regarding interestingness or importance Formal verification (such

as theorem provers of some sort) might some day be able to make such de-cisions

With a complete set of variables and their ranges as described in chapter

2, it is possible to arrive at an understanding of the overall size of the func-tional space of a given target Because these variables imply all of the mechanisms (elements) that use them, they may be said to represent the functional space completely This means that it is now possible to gain a complete characterization of the functional space of the target

3.2 Counting Function Points

The previous chapter explained that each unique combination of values of standard variables determines a single function point within the functional space In other words, a function point is represented by a tuple corre-sponding to the location of a point in some large multi-dimensional space, providing us with the coordinates of the point in this space It is possible to

arrive at an exact count P of these points by considering each category of

standard variable in turn and how it relates to the other categories of vari-ables Analysis begins with connectivity

3.2.1 Variables of Connectivity

Variables of connectivity are separated into two categories, variables of in-ternal connectivity and variables of exin-ternal connectivity Each variable has a finite range of discrete values that are countable This is true because these ranges are present in computer programs (verification software tools)

In many designs the ranges of variables of connectivity are independent

such that every value of some variable of connectivity k A can be associated

with each and every value of some other variable of connectivity k B So, if

for example a design has these two variables of internal connectivity k A and k B, then the number of points determined by only these two variables

is simply the product of the number of values in the range of each variable However, for many designs the range of values of certain variables are dependent upon the values of one or more other variables Some ranges are not simple finite sets of discrete values, but rather must be determined by methodically analyzing dependencies among the variables Consider the specification for a microprocessor that includes an optional cache control-ler capable of handling caches of varying size This defines two variables

of internal topology: CACHE_PRESENT and CACHE_SIZE The range of CACHE_PRESENT is [YES,NO] and the range of CACHE_SIZE is (for our example) [4K,8K,16K] The number of function points is not simply

2 ⋅ 3 = 6 due to the dependency of CACHE_SIZE on CACHE_PRESENT When CACHE_PRESENT is equal to NO, then there is no related value for CACHE_SIZE For that matter, one might argue that the variable CACHE_SIZE does not even exist when CACHE_PRESENT is equal to

NO For notational convenience, we will use the value of ∅ (the null value)

in the tuple when the value of the corresponding variable is undefined

So, in this example there are actually only 4 function points whose co-ordinates are:

• (NO,∅),

• (YES,4K),

• (YES,8K), and

• (YES,16K)

One further example illustrates how the count of function points grows quickly even for a design with relatively few variables of connectivity

Consider a simple design with 3 variables of internal connectivity (k i1 , k i2,

and k i3 ) and 2 variables of external connectivity (k e1 and k e2) Assume that all 5 variables are mutually independent

Let’s examine first how many different instances of the target can be

built Let k i1 have 2 values in its range, let k i2 have 3 values in its range,

and let k i3 have 4 values in its range Because all 3 variables are mutually independent, there are at most 2 ⋅ 3 ⋅ 4 = 24 different instances that can be built

Now consider how many different external contexts can surround each

of these 24 instances Let k e1 have 5 values in its range and let k e2 have 6 values in its range Again, because the variables are independent, there are

5 ⋅ 6 = 30 possible different contexts

42 Chapter 3 – Exploring Functional Space

Combining our 24 possible instances with our 30 possible contexts, we find that the subspace of connectivity contains 24 ⋅ 30 = 720 distinct func-tion points in our simple example In other words, it is possible to build

720 distinctly different systems Now let’s consider the general case

Consider the set X containing all of the values from the range of a

vari-able of connectivity (or any other category of varivari-able, for that matter) and also the null value ∅ The zero norm of X, denoted X , is the number of

non-zero elements of X This zero norm is also known as the Hamming weight of X For the purposes of analyzing functional spaces, X is

de-fined as the number of non-null elements of X

Assume that there are M different variables in k i (variables of internal

connectivity) and N different variables in k e (variables of external

connec-tivity) To arrive at the maximum number of function points P k in the space of connectivity, we count the number of values in the range of each variable and then take the product of all of these counts Mathematically this is stated as:

m =1

M

∏ and

n =1

N

∏

(3.1)

Then, the number of function points P k in the subspace of connectivity

of our design is at most the product of these two products:

Pk ≤ Pk i ⋅ Pk e. (3.2)

Any dependencies between ranges of variables will reduce the overall

number of possible instance-context pairs, so P k may usually be less than this upper bound To illustrate how these dependencies reduce the number

of function points, consider the first example in previous chapter that de-scribed the standard variables of external connectivity

In that example we had a simple central bus on which 3 different types

of agent could reside: from 1 to 8 processors, from 1 to 4 memory control-lers, and from 1 to 14 I/O controllers for a total of 16 agents The simple product of the counts of values in the ranges of the respective variables

yields a total of 448 function points (448 = 8 ⋅ 4 ⋅ 14) However, this count includes systems that violate the no-more-than-16-agents requirement Consider a system with one processor and one memory controller This particular system can include any of 1 to 14 I/O controllers for a total of 14 distinct contexts A system with one processor and two memory controllers can include any of 1 to 13 I/O controllers for a total of 13 distinct systems

Of course, it’s unlikely that actual systems corresponding to each and every one of these systems would be interesting in a commercial sense, but the point is that such systems are theoretically allowed within the ranges

defined by the requirements

Table 3.1 Number of possible contexts based on example in Fig 2.2

proc_agents mem_agents io_agents Possible

contexts

1

2

7

8

Continuing in this manner (see Table 3.1) we see that 50 (50 = 14 + 13 + 12 + 11) distinct single-processor systems can be assembled when only one processor is included Accounting for the dependency on the range for I/O controllers for the remaining possible N-processor systems, we see that only 288 possible systems can be assembled

This 288 is the actual number of function points in the space of connec-tivity, significantly fewer than the 448 calculated as the simple product, but still a large number of possible contexts in which the target of verifica-tion must operate correctly This is the challenge for the IP developer

44 Chapter 3 – Exploring Functional Space

In general it is not possible to rely on simple closed-form algebraic ex-pressions (such as in Eq 3.1 and Eq 3.2) to determine the number of func-tion points in the subspace of connectivity (or any other subspace, for that matter), but rather these function points must be counted algorithmically This is accomplished in a straightforward manner with nested for loops that examines each tuple of values of variable of internal connectivity (i.e., each instance) in turn and determining which tuples of values of variables

of external connectivity (i.e., contexts) are valid, just as was done to create Table 3.1

3.2.2 Variables of Activation (and other Time-variant Variables)

Variables of activation are categorized into three subsets, namely power, clocking, and reset For any valid system there may be multiple ways that the system can be activated, each described not by a single tuple of values

of these variables, but rather by sequences in time of changing values of these variables These sequences will be analyzed at length later in this chapter, but some preliminary analysis is in order here Consider the ex-ample of the multi-processor design that we have been using so far

In a system of multiple processors one of the processors is typically des-ignated as the monarch processor with the remaining processors desdes-ignated

as serfs Activation of the monarch precedes activation of the serfs, per-haps by holding each serf in its reset state until system resources such as memory are sufficiently initialized for the serfs to begin fetching and exe-cuting instructions on their own Also, some serf processors might not even be energized until they are needed And, a given serf might be de-activated to conserve power when it is not needed Each of these scenarios constitutes a different sequence of values of variables of activation, each sequence composed of multiple function points

Function points corresponding to time-variant variables such as those of activation are not counted as simply as time-invariant variables of connec-tivity Constructing the value transition graphs (defined later in this chap-ter) is necessary before the corresponding function points can be counted

Accounting for the 4 possible instances of the processor as determined

by CACHE_PRESENT and CACHE_SIZE, this yields 1,152 distinct sys-tems, all of which must be verified to function correctly At first glance this may appear to a rather dire situation for the verification engineer, but,

as will be seen later in this chapter, condensation in the functional space can provide significant practical advantages