Advanced Methods and Tools for ECG Data Analysis - Part 8 docx

These nonischemic changes may be significant, withbehavior similar to real transient ischemic or heart rate related ST segment episodes,and complicate manual and automated detection of t

Appendix 9A Description of the Karhunen-Lo`eve Transform 265

Consider a set of M-dimensional random vectors, {x}, the range of which is

part or all of P-dimensional Euclidean space An efficient eigenbasis to represent

{x} requires that the fewest eigenvectors be used to approximate {x} to a desired level of expected MSE Suppose that any sample pattern vector x= (x1, x2, , x M) T from this set belongs to L possible pattern classes {ωl, l = 1, 2, , L}, where the a priori probability of the occurrence of the lth class is p( ω l) Further assume that

each class is centralized by subtracting the mean µl of the random pattern vectors

xl in that class Denoting the centralized observation fromω lby zl, we write

while the coefficients clmsatisfy

and are mutually uncorrelated random coefficients for which

are their associated eigenvalues, whereρ mare the standard deviations of the cients Since the basis vectors are the eigenvectors of a real symmetric matrix, they

coeffi-are mutually orthonormal The eigenvectors Φm of the covariance matrix R and

their corresponding eigenvaluesλ mare found by solving

Denoting the KLT basis vectors (1,2, ,  M) in matrix notation , the KLT

transformation pair for pattern vector zl , and the coefficients of the expansion cl,may be expressed as

By this ordering, the optimal reduced KLT coordinate system is obtained in which

the first N coordinate coefficients contain most of the “information” about random

Appendix 9A Description of the Karhunen-Lo`eve Transform 267

represents the expected fraction of the total power of z associated with the vector Φk Next, if an approximation zl of zl is constructed according to (9A.13)

eigen-where N < M, the entropy function, given by

is minimized for all N This property guarantees that the expansion is of minimum

entropy and therefore a measure of minimum entropy or dispersion is associatedwith the coefficients of the expansion

We end the chapter with a description of specific performance measures and an uation protocol to assess the performance and robustness of ST change detectionalgorithms and analyzers Performance comparisons of a few recently developed STchange analyzers are presented It is assumed that the reader is familiar with thebackground presented in Chapter 9.

Typically ambulatory ECG data shows wide and significant (> 50 µV) transient

changes in amplitude of the ST segment level which are caused by ischemia, heartrate changes, and a variety of other reasons The major difficulties in automated STsegment analysis lie in the confounding effects of slow drifts (due to slow diurnalchanges), and nonischemic step-shape ST segment shifts which are axis-related (due

to shifts of the cardiac electrical axis) or conduction-change related (due to changes

in ventricular conduction) These nonischemic changes may be significant, withbehavior similar to real transient ischemic or heart rate related ST segment episodes,and complicate manual and automated detection of true ischemic ST episodes.The time-varying ST segment level due to clinically irrelevant nonischemic causesdefines the time-varying ST segment reference level This level must be tracked inorder to successfully detect transient ST segment episodes and then to distinguishnonischemic heart rate related ST episodes from clinically significant ischemic STepisodes

In choosing the ST segment change analysis recognition technique, the followingaspects and requirements should be taken into consideration:

1 Accurate QRS complex detection and beat classification is required Thepositioning of the fiducial point for each heartbeat should be accurate

2 Simultaneous analysis of two or more ECG leads offer the improvement ofanalysis accuracy in comparison to the single channel analysis with regard

to noise immunity and ST episode identification

269

3 The analysis technique should include robust preprocessing techniques, curate differentiating between nonnoisy and noisy events, and accurate STsegment level measurements.

ac-4 The representation technique should be able to encode as much tion as possible about the subtle structure of ST segment pattern vectors, ifpossible in terms of uncorrelated features

informa-5 The distribution of a large collection of ST segment features for normalheartbeats usually form a single cluster During ST change episodes, sig-nificant excursion of ST segment features over the feature space may beobserved The problem of detecting ST change episodes may be formulated

as a problem of detecting changes in nonstationary time series

6 The recognition technique should be able to efficiently and accurately rect the reference ST level by tracking the cluster of normal heartbeats due

cor-to the nonischemic slow drift of the ST segment level and due cor-to suddennonischemic step changes of the ST segment level

7 Classification between normal and deviating ST segments should take intoaccount interrecord and intrarecord variability of ST segment deviations

8 The recognition technique should be robust and able to detect transient STchange episodes and to differentiate between ischemic and heart rate related

ST episodes

9 The analysis technique may be required to function online in a single-scanmode with as short a decision delay as possible or in a multiscan mode (orperhaps using retrospective off-line analysis)

The development and evaluation of automated systems to detect transient ischemic

ST episodes has been most prominent since the release of the ESC DB [1], a ized reference database for development and assessment of transient ST segment and

standard-T wave change analyzers In the recent years, several excellent automated systemswere developed based on different approaches and techniques

Traditional time-domain analysis uses an ST segment function calculated as themagnitude of the ST segment vector determined from two ECG leads [2, 3], orthe filtered root mean square series of differences between the heartbeat ST seg-ment (or ST-T complex) and an average pattern segment [4], or ST segment levelfunction determined as ST segment amplitude measured at the heart rate adaptivedelays after the heartbeat fiducial point [5] The Karhunen-Lo`eve transform (KLT)approaches use sequential classification of ST segment KLT coefficients as normal

or deviating ones in the KLT feature space [6, 7] A technique for representingthe overall ST-T interval using KLT coefficients was proposed [8, 9] and used todetect ischemia by incorporating a filtered and differentiated KLT-coefficient timeseries [10] To improve the SNR of the estimation of the KLT coefficients, an adap-tive estimation was proposed [11] Another study showed that a global representa-tion of the entire ST-T complex appears to be more suitable than local measurementswhen studying the initial stages of myocardial ischemia [12] Neural network–based

10.2 Overview of ST Segment Analysis Approaches 271

approaches to classify ST segments as normal or ischemic include the use of a terpropagation algorithm [13], a backpropagation algorithm [14], a three-layerfeedforward paradigm [15], a bidirectional associative memory neural network[16], or an adaptive backpropagation algorithm [17] In these systems, a sequence

coun-of ST segments classified as ischemic forms an ischemic ST episode A variety

of neural network architectures to classify ST segments have been implemented,tested, and compared with competing alternatives [18] Architectures combiningprincipal component analysis techniques and neural networks were investigated aswell [18–20] Further efforts in seeking accurate and reliable neural network ar-chitecture to maximize the performance detecting ischemic cardiac heartbeats hasresulted in sophisticated architectures like nonlinear principal component analy-sis neural networks [21] and the network self-organizing map model [22, 23] Theself-organizing map model was successfully used to detect ischemic abnormalities inthe ECG without prior knowledge of normal and abnormal ECG morphology [24].Yet another system successfully detects ischemic ST episodes in long-duration ECGrecords using a feed-forward neural network and principal component analysis ofthe input to the network to achieve dimensionality reduction [25] Other automatedsystems to detect transient ischemic ST segment and T wave episodes employ fuzzylogic [26–28], wavelet transformation [29], a hidden Markov model approach [30],

or a knowledge-based technique [31] implemented in an expert system [32] ligent ischemia monitoring systems employ fuzzy logic [33] or describe ST-T trends

Intel-as changes in symbolic representations [34]

The detection of transient ST segment episodes is a problem of detecting eventsthat contain a time dimension There are insufficient distinct classes of ST segmentsand/or T waves with differing morphologies to allow the use of efficient classifica-tion techniques Some studies on the characterization of ST segment and T wavechanges [7, 35] have shown that morphology features of normal heartbeats form asingle cluster in the feature space This cluster of normal heartbeats is moving slowly

or in step shape fashion in the feature space due to slow nonischemic changes (drifts)

or due to sudden nonischemic changes (axis shifts) Ischemic and heart rate related

ST segment episodes are then defined as faster episodic trajectories (or excursions)

of morphology features out from and then back to the cluster of normal heartbeats.Therefore, it makes, sense to develop a technique which would efficiently trackthe cluster of normal heartbeats and would detect faster transient trajectories ofmorphology features

The majority of automated systems do not deal adequately (or even at all) withnonischemic events It was previously thought that the KLT-based systems and inparticular neural-network systems (since they extract information of morphologyfrom the entire ST segment), would separate subtle ischemia-related features of the

ST segment adequately from nonischemia related features Unfortunately, the cess of these techniques has been limited The problem of separating ischemic STepisodes from nonischemic ST segment events remains, in part due to the nonsta-tionarity of an ST segment morphology-feature time series, and the lack of a prioriknowledge of their distributions Furthermore, an insufficient number of nonis-chemic ST segment events present in the ESC DB prevents studying these events atlength and only short (biased or unrepresentative) segments of the database records(2 hours) have been used (since they were selected to be sufficiently “clean”) The

suc-other reference database for development and assessment of transient ST segmentchange analyzers, the LTST DB [35], contains long-duration (24-hour) records with

a large number of human-annotated ischemic and nonischemic ST segment events.Only a few automated systems deal explicitly with nonischemic events such asslow drifts and axis shifts One of the early systems [36] dealt with nonischemicevents by discriminating between “stable” and “unstable” ST segment baseline timeperiods and correcting the ST segment reference level for nonischemic shifts betweenstable periods Other systems employ ST segment level trajectory-recognition based

on heuristics in time domain [3], in the KLT feature space [7], or a combination oftraditional time-domain and KLT-based approaches [37] These systems are capable(to a certain extent) of detecting transient ST segment episodes and of tracking thetime-varying ST segment reference level

A few other systematic approaches to the problem of detecting body positionchanges which result in axis shifts have been made A technique based on a spatialapproach by estimating rotation angles of the electrical axis [38] and a techniqueusing a scalar-lead signal representation based on the KLT [39] were investigated.Another study used a measurement of R wave duration to identify changes in bodyposition [40] In all these investigations, the authors developed their own databaseswhich contain induced axis shifts

Currently developed ST episode detection systems are capable of detecting sient ST segment episodes which are ischemic or heart rate related ST episodes, butare not able to distinguish between them Automatic classification of these twotypes of episodes is an interesting challenge This task would require additionalanalysis of heart rate, original raw ST segment patterns, and clinical informationconcerning the patients A recognition algorithm would need to distinguish betweentypical ischemic and nonischemic ST segment morphology changes [35] These in-clude typical ischemic ST segment morphology changes (horizontal flattening, downsloping, scooping, elevation), which may or may not be accompanied by a change

tran-in heart rate, and typical heart rate related ST segment morphology changes (J potran-intdepression with positive slope, moving of the T wave into the ST segment, T wavepeaking, and parallel shifts of the ST segment compared with the reference or basal

ST segment), which are accompanied by an obligatory change in heart rate Theinclusion of clinical information also makes room for the development of sophisti-cated techniques leading to intelligent ischemia detection systems

Automated detection of transient ST segment changes requires: (1) accurate surement and tracking of ST segment levels and (2) detection of ST segment changeepisodes with correct identification of the beginning and end of each episode, andthe time and magnitude of the maximum ST deviation The main features of an

mea-ST change detection system may be: (1) the automatic tracking of the time-varying

ST segment reference level in the ST segment level time series of each ECG lead,

sl(i, k) (where i denotes the lead number and k denotes the sample number of the ST segment level time series) to construct the ST reference function, sr(i, k); (2) the ST deviation function, sd(i, k), in each lead which is constructed by taking the algebraic

10.3 Detection of Transient ST Change Episodes 273

difference between the ST level and ST reference function; (3) a combination of the

ST deviation functions from the leads into the ST detection function, D(k); and

(4) the automatic detection of transient ST episode

ST segment level from that of the ST segment level of a single reference heartbeatmeasured at the beginning of a record

The goal of the LTST DB is to be a representative research resource for velopment and evaluation of automated systems to detect transient ST segmentchanges, and for supporting basic research into the mechanisms and dynamics oftransient myocardial ischemia The LTST DB contains 86 two- and three-channel24-hour annotated AECG records of 80 patients (of varying lead combinations),collected during routine clinical practice ST segment annotations were made on av-erage heartbeats after considerable preprocessing A large number of nonischemic

de-ST segment events mixed with transient ischemic de-ST episodes allows development

of reliable and robust ST episode detection systems The ischemic and heart raterelated ST segment episodes were annotated in each lead separately according to anannotation protocol This protocol incorporates the ST segment deviation defined

as the algebraic difference between the ST segment level and the time-varying STsegment reference level (which was annotated throughout the records using local-reference annotations) The annotated events include: transient ischemic ST segmentepisodes, transient heart rate related nonischemic ST segment episodes, and nonis-chemic time-varying ST segment reference level trends due to slow drifts and stepchanges caused by axis shifts and conduction changes

The expert annotators of the ESC DB and LTST DB annotated transient STsegment episodes which satisfied the following clinically defined criteria:

1 An episode begins when the magnitude of the ST deviation first exceeds a

lower annotation detection threshold, Vlower= 50 µV.

2 The deviation then must reach or exceed an upper annotation detection

threshold Vupper throughout a continuous interval of at least Tmin s (theminimum duration of an ST episode)

3 The episode ends when the deviation becomes smaller than Vlower= 50 µV, provided that it does not exceed Vlowerin the following Tsep = 30 seconds(the interval separating consecutive ST episodes)

According to annotation protocol of the ESC DB, the values of the upper annotation

detection thresholds and minimum width of ST episodes are Vupper = 100 µV and

Tmin= 30 seconds The database contains 250 transient lead-independent ischemic

ST episodes combined using the logical OR function Episode annotations of the

LTST DB are available in three variant annotation protocols:

(A) Vupper= 75 µV, Tmin= 30 seconds;

(B) Vupper= 100 µV, Tmin= 30 seconds, equivalent to the protocol of the ESCDB;

(C) Vupper= 100 µV, Tmin= 60 seconds

According to the protocol A, the database contains 1,490 transient lead-independent

ischemic and heart rate related ST episodes combined using the logical OR

function Combining only ischemic ST segment changes yields 1,155 ischemic STepisodes

10.3.2 Correction of Reference ST Segment Level

Next we describe an efficient technique developed in [37] to correct the reference

ST segment level Using a combination of traditional time-domain and KLT-basedapproaches, the analyzer derives QRS complex and ST segment morphology fea-tures, and by mimicking human examination of the morphology-feature time seriesand their trends, tracks the time-varying ST segment reference level due to clinicallyirrelevant nonischemic causes These include slow drifts, axis shifts, and conductionchanges The analyzer estimates the slowly varying ST segment level trend, identifiesstep changes in the time series, and subtracts the ST segment reference level thusobtained from the ST segment level to obtain the measured ST segment deviationtime series that is suitable for detection of ST segment episodes

10.3.2.1 Estimation of ST Segment Reference Level TrendHuman experts track the slowly varying trend of ST segment level and skip the morerapid excursions during transient ST segment events Similarly, the analyzer [37] es-

timates the time-varying global and local ST segment reference level trend [srg(i, k) and srl(i, k), respectively], of the ST level functions, sl(i, k), by applying two moving-

average lowpass filters The ST level functions were obtained using preprocessing,exclusion of noisy outliers, resampling, and smoothing of the time series (as de-

scribed in the Chapter 9) The two moving-average lowpass filters are: hg, over 6hours and 40 minutes in duration estimating the global nonstationary mean of the

ST level function, and hl, over 5 minutes in duration estimating local excursions

of the ST level function Moving-average lowpass filters posses useful frequencycharacteristic which are simple to realize and computationally inexpensive The STreference function is estimated as follows:

sr (i, k)=

srg(i, k) : if|srg(i, k) − srl(i, k)| > Ks

where Ks = 50 µV is the “significance threshold” (i.e., the threshold to locate

significant excursions of the ST level function from its global trend, and is equivalent

to a lower annotation detection threshold, Vlower = 50µV) The moving-average

10.3 Detection of Transient ST Change Episodes 275

filter hg estimates the global nonstationary trend of the ST level function, s(i, k) The sr (i, k) is composed from the srg(i, k) and an excursion of the ST segment level

indicates that a transient ST segment episode has occurred at this time

10.3.2.2 Detection of Sudden Step Changes

A human expert considers each step change of an ST level function (which is companied by a step change in the QRS complex morphology, and preceded andfollowed by a stable interval with no change of the QRS complex and ST segmentmorphology) as a step-shape path of the ST segment reference level To detect step

ac-changes, the analyzer [37] uses the ST level function, sl(i, k), and the first-order

Ma-halanobis distance functions of KLT-coefficient morphology-feature vectors of the

QRS complex, d(yqrs(k), yqrs(1)), and of the ST segment, d(yst(k), yst(1)) Besides

a step change, there has to be a “flat” interval of the three functions before andafter the step change In each of the three functions, the analyzer searches first for

a flat interval of 216 seconds in length, which has to have its mean absolute

devia-tion from its own mean value less than Kf= 20 µV for the ST level function (and

less than = 0.33 SD for both the QRS and ST Mahalanobis distance functions).

Such a flat interval has to be followed by a step change, which is characterized bythe moving average value over 72 seconds in length and has to change for more

than Ks = 50 µV for the ST level function (and for more than qrs= 0.5 SD and

st = 0.4 SD for QRS and ST Mahalanobis distance functions) within the next

144 seconds in length This step change has to be followed by another flat interval

in each of the functions, defined as for the first flat interval

The operation used to detect step changes actually computes the derivative

of the three functions and therefore behaves like a band-pass filter extracting rapidslopes while rejecting spikes and noises (by attenuating high frequencies) that might

be present in the three functions In the intervals surrounding each step changedetected, the ST reference function is updated as follows:

first 60 minutes of the data segment shown, the ST reference function, sr(1, k), is estimated by the global ST reference level trend, srg(1, k), and after that by the ST level function, sl(1, k), because of detected axis shifts in the region.

10.3.3 Procedure to Detect ST Change Episodes

After estimating the reference ST level, the ST deviation function, sd(i, k), for each

ECG lead can be derived as algebraic difference between the ST level function and

ST reference function:

sd(i, k) = sl(i, k) − sr(i, k) (10.3)

Figure 10.1 Example of tracking the reference ST level, deriving of ST reference, ST deviation, and

ST detection function, and of detecting ST change episodes in the record s30661 of the LTST DB using the system in [37] A 3-hour data segment is shown During the ischemic ST episode, the ST

reference function, sr(1, k), is estimated by the global ST reference level trend, srg(1, k), and after that

by the ST level function, sl(1, k), due to detected axis shifts The arrows mark the axis shifts From top

to bottom plotted in time scale: heart rate; ST level function, sl(1, k); ST reference function, sr(1, k); ST deviation function, sd(1, k); ST detection function, D(k), (resolution: 100 µV); and lead-independent

combined ischemic ST episode annotation stream derived by expert annotators (lower line) and ST episodes detected by the analyzer (upper line).

Finally, the ST detection function, D(k), is derived as a combination of ST deviation

functions from the ECG leads:

An important part of each ST segment analyzer is an algorithm to automaticallydetect and annotate significant transient ST segment episodes The algorithm has

to classify sequentially samples of the ST detection function, D(k), to normal and

deviating ones, has to model human-defined timing criteria for the identification of

ST episodes, and must annotate the beginnings, extrema, and ends of transient STsegment episodes Figure 10.2 symbolically summarizes such an algorithm whichstrictly follows the human-expert timing criteria of the annotation protocols of theESC DB and LTST DB with two arbitrary amplitude thresholds Samples of the

detection function, D(k), of the algorithm may be, for example: a time-domain ture [e.g., the ST segment deviation, sd(i, k)]; a combination of features (e.g., the

fea-Euclidean distance of ST segment deviations from the leads); or the Mahalanobis

distance function d or d2of the ST segment KLT-coefficient morphology-feature tors (The detection algorithm assumes positive values of the detection function.)

vec-Consecutive samples of the detection function, D(k), are classified according to lower and upper feature space boundaries, Ulower and Uupper, which correspond to

human-expert reference annotation thresholds Vlowerand Vupper Different

annota-tion protocols can be assumed by selecting two thresholds, Ulower and Uupper, and

10.3 Detection of Transient ST Change Episodes 277

Figure 10.2 Algorithm for detecting transient ischemic and heart rate related ST episodes in the ST

segment detection function, D(k) On input the algorithm accepts feature space boundaries, Ulower

and Uupper, and minimum width of ST episodes, Tmin , according to selected criteria for significant

ST episodes On output the algorithm returns the number of detected transient ST episodes, Nepis ,

and arrays of times of beginnings, Tbeg(Nepis), extrema, Text(Nepis), and ends Tend(Nepis ), of detected episodes.

a proper minimum width of ST episodes, Tmin A segmentation logic of the rithm uses segmentation rules which follow human-expert defined criteria for iden-tifying transient ST segment episodes The logic operates in a sequential manner

algo-and identifies segments of the D(k) belonging to normal segments algo-and separating

ST episodes, transition segments containing the exact beginning of the ST episode,segments which are a part of ST episodes, and transition segments containing theexact end of the ST episode Each ST segment episode is then defined between theexact beginning and the exact end of the episode

Besides tracking the reference ST level using the analyzer [37], Figure 10.1 alsoshows an example of detecting ST change episodes when using the detection algo-rithm from Figure 10.2 The detection function in the example uses the Euclidean

distance of the sd(i, k) from the ECG leads The algorithm correctly detected the

ischemic ST episode present in the first part of the data segment shown

The evaluation of an ST detection algorithm or analyzer should answer the followingquestions:

• How well are ST episodes detected?

• How well are ischemic and nonischemic heart rate related ST episodesdifferentiated?

• How reliably are ST episode or ischemic ST episode duration measured?

• How accurately are ST deviations measured?

• How well will the ST analyzer perform in the real world?

In this section we describe performance measures and an evaluation protocol forassessing the performance of ST algorithms and analyzers according to these eval-uation questions

10.4.1 Performance Measures

Three performance measures are commonly used to assess an analyzer performance:

sensitivity, specificity, and positive predictive accuracy Sensitivity, Se, the ratio of the number of correctly detected events, TP (true positives), to the total number of

where FP (false positives) is the number of falsely detected events Positive predictive

accuracy,+P, (or just positive predictivity) is the ratio of the number of correctly detected events, TP, to the total number of events detected by the analyzer and is

ap-10.4 Performance Evaluation of ST Analyzers 279

probability of true positives:

that is, the conditional probability of the decision of EVENT given that the event

occurred Specificity approximates the conditional probability of true negatives:

that is, the conditional probability of the decision of NONEVENT given that the nonevent occurred Positive predictivity approximates the posterior probability of true positives [i.e., the posterior probability that event occurred given the decision (evidence) of EVENT]:

p(event |EVENT) = p(EVENT |event) p(event)

In many detection problems, nonevents cannot be counted, so that the number of

true negatives, TN, is undefined In such problems, the commonly used performance measures are sensitivity, Se, the proportion of events which were de-

detector-tected, and positive predictivity,+P, the proportion of detections which were events,

or the accuracy of classifying detected events

To evaluate an analyzer’s ability to detect significant (> 50 µV) ischemic ST

episodes (characterized by the beginning, the extrema, and by the end), it is sary to match reference ST episodes with analyzer-annotated ST episodes With amatching criteria, the concept of sensitivity (the fraction of correctly detected events)and positive predictivity (the fraction of detections which are events) are applicable,while specificity (the fraction of rejections which are correct) is not applicable, since

neces-the number of nonevents, TN, is undefined We describe next particular sensitivity

and positive predictivity metrics which are helpful in quantifying performance Theperformance measures to assess the accuracy of detecting ischemic ST episodes andtotal ischemic time are based on the concepts of matching and overlap betweenreference and analyzer-annotated episodes

10.4.1.1 Detection of ST EpisodeTransient ST segment episodes (the events of interest) are characterized by: (1)number, (2) length, and (3) extrema deviation When evaluating multichannel ST-analyzer performance, the ST annotation stream for all leads must be combined

into one reference stream using a logical OR function The fact that at any given

time there is either an ST episode or an interval with no ST deviation impliesthe use of two-by-two performance evaluation matrices We further assume thatall ST episodes are equally important Evaluation of ST episode detection ana-lyzers consists of comparing analyzer-annotated episodes with reference-annotatedepisodes There is not a one-to-one correspondence between analyzer- and reference-annotated episodes; the episodes from the two groups may differ considerably inlength Furthermore, nonevents cannot be counted

Figure 10.3 Matching criteria defined for a correctly detected ST episode, tps , and a missed ST

episode, fn.

The performance measures to assess ability to detect ST episodes depend on theconcept of matching [42–44] A match of a reference or analyzer-annotated episodeoccurs when the period of mutual overlap includes at least a certain portion of thelength of the episode according to the defining annotations In measuring sensitivity(see Figure 10.3), matching of a reference ST episode occurs when the period ofoverlap includes at least one of the extrema of the reference ST episode, or at leastone-half of the length of the reference ST episode In measuring positive predictivity(see Figure 10.4), the matching of analyzer-annotated ST episodes occurs when theperiod of overlap includes the extrema of the analyzer-annotated ST episode, or atleast one-half of the length of the analyzer-annotated ST episode

The sensitivity matrix (see Figure 10.5, left) summarizes how the reference STepisodes were labeled by the analyzer (i.e., how many of the reference ST episodes

were detected, TPS, and how many were missed, FN) The positive predictivity

matrix (Figure 10.5, right) summarizes how many of the analyzer-annotated ST

episodes were actually ST episodes, TPP, and how many were falsely detected, FP.

ST episode detection sensitivity, SESe, an estimate of the likelihood of detecting

an ST episode, is defined as

The denominator quantifies the number of reference ST episodes, TPSis the number

of matching episodes, and FN is the number of nonmatching episodes where the

defining annotations are the reference annotations

ST episode detection positive predictivity, SE + P, an estimate of the likelihood

that a detection is a true ST episode, is defined as

SE + P = TPP

Figure 10.4 Matching criteria defined for an analyzer-annotated ST episode, which is actually an

ST episode, tp , and a falsely detected ST episode, fp.

10.4 Performance Evaluation of ST Analyzers 281

Figure 10.5 ST episode sensitivity matrix (left) and ST episode positive predictivity matrix (right).

Not epis indicates the absence of an ST episode.

The denominator quantifies the number of ST episodes annotated by the analyzer,

TPP is the number of matching episodes, and FP is the number of nonmatching

episodes, where the defining annotations are the analyzer annotations

10.4.1.2 Differentiation Between Ischemic and Heart Rate Related ST Episodes

In differentiating ischemic and nonischemic heart rate related ST episodes, we sumed that at any given time there is only one type of episode: either ischemic,nonischemic heart rate related, or an interval without significant ST deviation.This implies three-by-three performance evaluation matrices (see Figure 10.6) Eachreference- and analyzer-annotated episode is submitted to the extended matchingtest [45] The test is the same as defined previously for ST episodes, but extended

as-in the sense that matchas-ing of an episode (ischemic or heart rate related) occurswhen the episode is sufficiently and uniquely overlapped by ischemic or by heartrate related ST episodes The criteria of extended matching test to determine the

status of uth reference (truth) ST episode (ischemic or heart rate related), SR(u), are

summarized in the following:

if match with ischemic analyzer-annotated ST episodes then

SR(u) = ischemic;

else if match with heart rate related analyzer-annotated ST episodes then

SR(u) = heart rate related;

else

SR(u) = missed;

endif

Figure 10.6 Performance matrices assessing the ability of an ST episode detection analyzer to

dif-ferentiate ischemic (Isch) and nonischemic heart rate related (HR rel) ST episodes.

Similarly, the criteria of extended matching test to determine the status of vth analyzer-annotated (analyzer) ST episode (ischemic or heart rate related), SA(v),

are summarized in the following:

if match with ischemic reference-annotated ST episodes then

SA(v) = ischemic;

else if match with heart rate related reference-annotated ST episodes then

SA(v) = heart rate related;

of missed ischemic and heart rate related episodes, respectively The positive dictivity matrix (Figure 10.6, right) describes how many of the analyzer’s ischemic,

pre-G, and heart rate related, J , ST episode detections were actually ischemic and heart rate related episodes H is the number of the analyzer’s heart rate related episode detections which actually are reference ischemic episodes, and I is the number of

the analyzer’s ischemic episode detections which actually are reference heart rate

related episodes K and L are the numbers of falsely detected ischemic and heart

rate related episodes, respectively

Furthermore, if we consider both ischemic and heart rate related ST changestogether as ST change episodes of unique type, then the performance matrices can

easily be reduced back to two-by-two, with: TPS = A + B + D + E, TPP = G +

H + I + J , FN = C + F , and FP = K + L, yielding the performance matrices in

Figure 10.5 Since the events of clinical interest are the ischemic ST episodes, wecan further consider all nonischemic heart rate related ST episodes as episodes of

no deviation This consideration yields: TPS = A, TPP = G, FN = B + C, and

FP = I + K, and leads to the ischemic ST episode detection sensitivity, IESe, and ischemic ST episode detection positive predictivity, IE+P, which are defined in the same manner as the SESe and SE+P in (10.11) and (10.12).

10.4.1.3 Measurement of ST Episode Durations

ST episode duration detection sensitivity, SD Se, is the estimate of the accuracy with

which an analyzer can measure the duration of ST episodes within the observation

period SD Se is defined as the fraction of true ST episode durations detected and

is given by

SD Se= SDR∧A

SDR

(10.13)

ST episode duration detection positive predictivity, SD + P, is defined as the fraction

of analyzer-annotated ST episode durations which are true ST episodes and is given

10.4 Performance Evaluation of ST Analyzers 283

by

SD + P = SDR∧A

SDA

(10.14)

where SDR∧Ais the total duration of analyzer-annotated ST episodes which overlaps

reference ST episodes, and SDRand SDA are the total durations of reference- andanalyzer-annotated ST episodes, respectively [42–44] Similarly, ischemia duration

detection sensitivity, ID Se, and ischemia duration detection positive predictivity,

ID + P, can be defined using total duration of analyzer-annotated ischemia, IDA,

which overlaps reference ischemia, IDR, and their overlap, IDR∧A

10.4.1.4 Measurement of ST Segment DeviationsAccuracy of ST-deviation measurement of the extrema of ST episodes is usually sum-marized by a scatter plot of reference versus test measurements Such a scatter plotpermits rapid visual assessment of any systematic measurement bias, nonlinearity,

or unreliable performance of the ST deviation measurement analyzer Other usefulsummary statistics are: mean error between the analyzer and reference measure-ments, standard deviation of errors, correlation coefficient, and linear regression.These statistics do not distinguish between errors resulting from bias or nonlin-earity and errors resulting from poor noise tolerance or unreliable measurementtechniques Other more robust and informative statistics in the presence of outliersare: the value of error, which 95% of the measurements do not exceed, and thepercentage of measurements for which the absolute difference between the analyzerand reference measurement is greater than 100 µV [42–44].

10.4.1.5 Predicting Real-World Performance

In predicting the analyzer’s performance in the real world, it is important to use a testdatabase, which was not used for development In addition, a second-order aggre-gate gross statistic which weights each event equally by pooling all the events over allrecords together and models how the analyzer behaves on a large number of events,and a second-order aggregate average statistic which weights each record equallyand models how the analyzer behaves on randomly chosen records, are applicable

If the database is so small that it could not be divided into development and testsubsets, or if additional data is not available, the bootstrap technique [46, 47] is use-ful for predicting the analyzer’s performance in the real world The method assumesthat the database is a well-chosen representative subset of examples for a problemdomain and does not require any assumption about the distribution of the data

By this technique, many new databases are chosen at random (with replacement;that is, a newly chosen record put back) from the original database, and perfor-mance statistics are derived for each new database The mean, standard deviationand median of expected performance, as well as the minimum expected perfor-mance (5% confidence limits), can be estimated from the distributions of the per-formance statistics Due to relative complexity of the performance measures and ofthe evaluation protocol, an automated tool to objectively evaluate and compare theperformance and robustness of transient ST episode detection analyzers is desirable

The open-source tool EVAL ST [45] provides first (record-by-record) and order (aggregate gross and average) performance statistics for evaluation and com-parison of transient ST episode detection analyzers Inputs to the tool are ST segmentannotation streams of a reference database (e.g., the ESC DB or the LTST DB) and

second-ST segment annotation streams of the evaluated analyzers The tool allows ing the accuracy of: (1) detecting transient ST episodes, (2) distinguishing betweenischemic and nonischemic heart rate related ST episodes, (3) measuring ST episodedurations and ischemic ST episode durations, and (4) measuring ST segment devia-tions The tool also generates performance distributions using a bootstrap statisticaltechnique for predicting real-world clinical performance and robustness A graphicuser interface module of the tool provides display of all evaluation results The toolhas been made freely available on http://www.physionet.org/physiotools/eval st/,the PhysioNet Web site [48]

assess-10.4.2 Comparison of Performance of ST Analyzers

Table 10.1 comparatively summarizes the performances of transient ST episodedetection systems developed and tested on the the ESC DB and the LTST DB Onlythose systems are compared which were developed and tested using the originalESC DB and original LTST DB annotations These systems incorporate time-domainanalysis [3–5], the KLT approach [7], a combination of time-domain analysis andKLT approach [37], a neural network approach [17], and a combination of theKLT transform and a neural network approach [18] These systems were evaluatedusing commonly accepted performance measures [42, 49, 50] (A slightly modifiedmatching test between analyzer’s and reference ST episodes were used in [17].) The

Table 10.1 Comparison of Performance of Transient ST Episode Detection Systems Developed and

Tested on the ESC DB and LTST DB

ESC DB LTST DB (Protocol B)

S E [%] S D [%] S E [%] S D [%] System, Technique Se +P Se +P Se +P Se +P

[a] 84.7 86.1 75.3 68.2 – – – – [5], Time domain [g] 79.2 81.4 – – – – – –

[7], KLT approach [g] 85.2 86.2 75.8 78.0 –> 77.0 58.8 48.5 47.8

[a] 87.1 87.7 78.2 74.1 –> 74.0 61.4 54.8 58.4 [37], Time domain, KLT [g] 77.2 86.3 67.5 69.2 <– 79.6 78.3 68.4 67.3

[a] 81.3 89.2 77.6 68.9 <– 78.9 80.7 73.1 74.9 [17], Neural net [g] 85.0 68.7 73.0 69.5 – – – –

[a] 88.6 78.4 72.2 67.5 – – – – [18], Neural net, KLT [g] – – – – – – – –