Advanced Verification Flow part 1

This test plan forms the fundamental framework of the functional verification environment.. These steps include Hardware Acceleration, Hardware Emulation and Assertion based Verification

[ Team LiB ]

15.1 Traditional Verification Flow

A traditional verification flow consisting of certain standard components is illustrated in Figure 15-1 This flow addresses only the verification perspective It assumes that logic design is done separately

Figure 15-1 Traditional Verification Flow

As shown in Figure 15-1, the traditional verification flow consists of the following steps:

1 The chip architect first needs to create a design specification In order to create a good specification, an analysis of architectural trade-offs has to be performed so that the best possible architecture can be chosen This is usually done by

simulating architectural models of the design At the end of this step, the design specification is complete

2 When the specification is ready, a functional test plan is created based on the design specification This test plan forms the fundamental framework of the

functional verification environment Based on the test plan, test vectors are applied

to the design-under-test (DUT), which is written in Verilog HDL Functional test environments are needed to apply these test vectors There are many tools

available for generating and apply test vectors These tools also allow the efficient creation of test environments

3 The DUT is then simulated using traditional software simulators (The DUT is normally created by logic designers Verification engineers simulate the DUT.)

4 The output is then analyzed and checked against the expected results This can be done manually using waveform viewers and debugging tools Alternately, analysis can be automated by the test environment checking the output of the DUT or by parsing the log files using a language like PERL In addition, coverage results are analyzed to ensure that the tests have exercised the design thoroughly and that the verification goals are met If the output matches the expected results and the

coverage goals are met, then the verification is complete

5 Optionally, additional steps can be taken to decrease the risk of a future design re-spin These steps include Hardware Acceleration, Hardware Emulation and

Assertion based Verification

Earlier, each step in the traditional verification flow was accomplished with Verilog HDL Though Verilog HDL remains the dominant method for creating the DUT, many advances have occurred in the other steps of the verification flow The following sections describe these advances in detail

15.1.1 Architectural Modeling

This stage includes design exploration by the architects The initial model typically does not capture exact design behavior, except to the extent required for the initial design decisions For example, a fundamental algorithm like an MPEG decoder might be

implemented, but the processor to memory bandwidth is not specified The architect tries out several different variations of the model and make some fundamental decisions about the system These decisions may include number of processors, algorithms implemented

in hardware, memory architecture, and so on These trade-offs will affect the eventual implementation of the target design

Architectural models are often written using C and C++ Though C++ has the advantage

of object oriented constructs, it does not implement concepts such as parallelism and timing that were found in HDLs Thus, creators of architectural models have to

implement these concepts in their models This is very cumbersome, resulting in long development times for architectural models

To solve this problem, architectural modeling languages were invented These languages

have both the object oriented constructs found in C++ as well as parallelism and timing constructs found in HDLs Thus, they are well-suited for high-level architectural models

A likely advancement in the future is the design of chips at the architectural modeling level rather than at the RTL level High-level synthesis tools will convert architectural models to Verilog RTL design implementations based on the trade-off inputs These RTL designs can then go through the standard ASIC design and verification flow Figure 15-2 shows an example of such a flow

Figure 15-2 Architectural Modeling

Appendix E, Verilog Tidbits, contains further information on popular architectural

modeling languages

15.1.2 Functional Verification Environment

The functional verification of a chip can be divided into three phases

• Block level verification: Block level verification is usually done by the block

designer using Verilog for both design and verification A number of simple test cases are executed to ensure that the block functions well enough for chip

integration

• Full ChipVerification: The goal of full chip verification is to ensure that all the

features of the full chip described in the functional test plan are covered

• Extended Verification: The objective of the extended verification is to find all

corner case bugs in the design This phase of verification is lengthy since the set of tests is not predetermined and it may continue past tape-out

During the functional verification phase, a combination of directed and random

simulation is used Directed tests are written by the verification engineers to test a

specific behavior of the design They may use random data, but the sequence of events are predetermined Random sequences of legal input transactions are used towards the end of functional verifcation and during the extended verification phases in order to simulate corner cases which the designer may have missed

As Verilog HDL became popular, designers[1] started using Verilog HDL to both the DUT and its surrounding functional verification environment In a typical HDL-based verification environment,

[1]

In this chapter, the words "designer" and "verification engineer" have been used

interchangeably This is because logic designers perform block level verification and are often involved in the full chip verification process

• The testbench consisted of HDL procedures that wrote data to the DUT or read

data from it

• The tests, which called the testbench procedures in sequence to apply manually

selected input stimuli to the DUT and checked the results, were directed only towards specific features of the design as described in the functional test plan However, as design sizes exceeded million gates, this approach became less effective because

• The tests became harder and more time consuming to write because of decreasing

controllability of the design

• Verifying correct behavior became difficult due to decreasing observability into

internal design states

• The tests became difficult to read and maintain

• There were too many corner cases for the available labor

• Multiple environments became difficult to create and maintain because they used

little shared code

To make the test environment more reusable and readable, verification engineers needed

to write the tests and the test environment code in an object oriented programming

language High-Level Verification Languages (HVLs) were created to address this need Appendix E, Verilog Tidbits, contains further information on popular HVLs

HVLs are powerful because they combine the object oriented approach of C++ with the

parallelism and timing constructs in HDLs and are thus best suited for verification HVLs also help in the automatic generation of test stimuli and provide an integrated

environment for functional verification, including input drivers, output drivers, data checking, protocol checking, and coverage Thus, HVLs maximize productivity for

creating and maintaining verification environments

Figure 15-3 shows the various components of a typical functional verification

environment HVLs greatly improve the designer's ability to create and maintain each test component Note that Verilog HDL is still the primary method of creating a DUT

Figure 15-3 Components of a Functional Verification Environment

In an HVL-based methodology, the verification components are simulated in the HVL simulator and the DUT is simulated with a Verilog simulator The HVL simulator and the Verilog simulator interact with each other to produce the simulation results Figure 15-4 shows an example of such an interaction The HVL simulator and Verilog simulator are run as two separate processes and communicate through the Verilog PLI interface The HVL simulator is primarily responsible for all verification components, including test generation, input driver, output receiver, data checker, protocol checker, and coverage analyzer The Verilog simulator is responsible for simulating the DUT

Figure 15-4 Interaction between HVL and Verilog Simulators

The future trend in HVLs is to apply acceleration techniques to HVL simulators to

greatly speed up the simulations These acceleration techniques are similar to those used for Verilog HDL simulators and are discussed in Section 15.1.3, Simulation

15.1.3 Simulation

There are three ways to simulate a design: software simulation, hardware acceleration, and hardware emulation

Software Simulation

Software simulators were typically used to run Verilog HDL-based designs Software simulators run on a generic computer or server They load the Verilog HDL code and simulate the behavior in software Appendix E, Verilog Tidbits, contains further

information on popular software simulators

However, when designs started exceeding one million gates, software simulations began

to consume large amounts of time and became a bottleneck in the verification process Thus, various techniques emerged to accelerate these simulations Two techniques, hardware acceleration and emulation, were invented

Hardware Acceleration

Hardware acceleration is used to speed up existing simulations and to run long sequences

of random transactions during functional and extended verification phases

In this technique, the Verilog HDL-based design is mapped onto a reconfigurable

hardware box The design is then run on the hardware box to produce simulation results Hardware acceleration[2] can often accelerate simulations by two to three orders of

magnitude

[2]

Also know as "Simulation Acceleration."

Hardware accelerators can be FPGA-based or processor-based The simulation is divided between the software simulator, which simulates all Verilog HDL code that is not

synthesizable, and the hardware accelerator, which simulates everything that is

synthesizable

Figure 15-5 shows the verification methodology with a hardware accelerator

Figure 15-5 Hardware Acceleration

Verification components may be simulated using a Verilog simulator or an HVL

simulator The simulator and the hardware accelerator interact with each other to produce

results

Hardware accelerators can cut simulation times from a matter of days to a few hours Therefore, they can greatly shorten the verification timeline However, they are expensive and need significant set-up time Another drawback is that they usually require long compilation times, which means that they are most useful only for long regression

simulations As a result, smaller designs still employ software simulation as the

simulation technique of choice

Appendix E, Verilog Tidbits, contains further information on popular hardware

accelerators

Hardware Emulation

Hardware emulation[3] is used to verify the design in a real life environment with real system software running on the system HW emulation is used during the extended

verification phase since the design must be pretty stable

[3]

Also know as "In-Circuit Emulation."

One of the major benefits of hardware emulation is that hardware software integration can start before the actual hardware is available, thus saving time in the schedule By running real-life software, conditions that are very difficult to set up in a simulation environment can be tested

When the design is complete, software engineers often want to run their software on the design before the design is realized on a chip Here are a few examples of what the

designer of a chip might want to do before a chip is sent for fabrication:

1 Designers of a microprocessor want to try booting the UNIX operating system

2 Designers of an MPEG decoder want to have live frames decoded and shown on a screen

3 Designers of a graphics chip want actual frame renderings to show up on the screen in real time

Running live systems with the design is an important verification step that reduces the possibility of bugs and a design re-spin However, software simulators and hardware accelerators cannot be used for this purpose because they are too slow and do not have the necessary hooks to run a live system For example, to boot UNIX with a software simulation of a design may take many years Hardware emulation can boot UNIX in a few hours

Figure 15-6 shows the setup of a typical emulation system Emulation is done so that the

software application runs exactly as it would on the real chip in a real circuit The

software application is oblivious to the fact that it is running on an emulator rather than the actual chip

Figure 15-6 Hardware Emulation

Hardware emulators typically run at megahertz speeds However, they are very expensive and require significant setup time As a result, smaller designs still employ software simulation as the simulation technique of choice

Appendix E, Verilog Tidbits, contains further information on popular hardware

emulators

15.1.4 Analysis

An important step in the traditional verification flow is to analyze the design to check the following items:

1 Was the data received equal to the expected data?

2 Was the data received correctly according to the interface protocol?

To analyze the correctness of the data value and data protocol, various methods are used

1 Waveform Viewers are used to see the dump files The designer visually goes through the dump files from various tests and ensures that the data value and the data protocol are both correct

2 Log Files contain traces of the simulation run The designer visually looks at the log files from various tests and determines the correctness of the data value and the data protocol based on the simulation messages

These methods are extremely tedious and time-consuming Every time a test is run, the designers has to manually look through the dump files and log files This method breaks down when a large number of simulation runs needs to be analyzed Therefore, it is advisable to make your test environment self-checking Two components are required for building a self-checking test environment:

1 Data Checker

2 Protocol Checker

Data checkers compare each value output from the simulation and check the value on-the-fly against the expected output If there is a mismatch, the simulation can be stopped immediately to display an error message If there are no error messages, the simulation is deemed to complete successfully Scoreboards are often used to implement data checkers Scoreboards are often used to indicate the completion transactions and verify that data is received on the corrected interface Scoreboards also ensure that data is not lost in the DUT, even if the protocol is followed, and that the data received is correct

Protocol checkers check on-the-fly whether the data protocol is followed at each input and output interface If there is a violation of protocol, the simulation can be stopped immediately to display an error message If there are no error messages, the simulation is deemed to complete successfully

A self-checking methodology allows the designer to run thousands of tests without

having to analyze each test for correctness If there is a failure, the designer can probe further into the dump files and the log files to determine the cause of the error

15.1.5 Coverage

Coverage helps the designer determine when verification is complete Various methods have been developed and used to quantify the verification progress There are two types

of coverage: structural and functional