The applications of HOS on biomedical signals are clustered according to the HOS property they most rely on, that is, (1) the ability to describe non-Gaussian processes and preserve phase, (2) the Gaussian noise immunity, and (3) the ability to characterize nonlinearities.
In the first class are the works of References 40 to 42, where ultrasound imaging distortions are estimated from the ultrasound echo and subsequently compensated for, to improve the diagnostic quality of the image. HOS have been used in modeling the ultrasound radio-frequency (RF) echo [46], where schemes for estimation of resolvable periodicity as well as correlations among nonresolvable scatters have been proposed. The “tissue color,” a quantity that describes the scatterer spatial correlations, and which can be obtained from the HOS of the RF echo, has been proposed [43] as a tissue characterization feature. The skewness and kurtosis of mammogram images have been proposed in Reference 44 as a tool for detecting microcalcifications in mammograms.
In the second class are methods that process multicomponent biomedical signals, treating one com- ponent as the signal of interest and the rest as noise. HOS have been used in Reference 45 to process lung sounds in order to suppress sound originating from the heart, and in Reference 46 to detect human afferent nerve signals, which are usually observed in very poor signal-to-noise ratio conditions.
Most of the HOS applications in the biomedical area belong to the third class, and usually invest- igate the formation and the nature of frequencies present in signals through the presence of quadratic phase coupling (QPC). The bispectrum has been applied in EEG signals of the rat during various vigil- ance states [47], where QPC between specific frequencies was observed. QPC changes in auditory evoked potentials of healthy subjects and subjects with Alzheimer’s dementia has been reported [48]. Visual evoked potentials have been analyzed via the bispectrum in References 49 to 51. Bispectral analysis of interactions between electrocerebral activity resulting from stimulation of the left and right visual fields revealed nonlinear interactions between visual fields [52]. The bispectrum has been used in the ana- lysis of electromyogram recruitment [53], and in defining the pattern of summation of muscle fiber twitches in the surface mechanomyogram generation [54]. QPC was also observed in the EEG of humans [55,56].
In the sequel, presented in some detail are the application of HOS on improving the resolution of ultrasound images, a topic that has recently attracted a lot of attention.
Ultrasonic imaging is a valuable tool in the diagnosis of tumors of soft tissues. Some of the distinct- ive features of ultrasound are its safe, nonionizing acoustic radiation, and its wide availability as a low cost, portable equipment. The major drawback that limits the use of ultrasound images in certain cases (e.g., breast imaging) is poor resolution. In B-Scan images, the resolution is compromised due to: (1) the finite bandwidth of the ultrasonic transducer, (2) the non-negligible beam width, and (3) phase aber- rations and velocity variations arising from acoustic inhomogeneity of tissues themselves. The observed ultrasound image can be considered as a distorted version of the true tissue information. Along the axial
direction the distortion is dominated by the pulse-echo wavelet of the imaging system, while along the lateral direction the distortion is mainly due to finite-width lateral beam profile. Provided that these distortions are known in advance, or non-invasively measurable, their effects can be compensated for in the observed image. With propagation in tissue, however, both axial and lateral distortions change due to the inhomogeneities of the media and the geometry of the imaging system. They also change among different tissue types and individuals. Distortions measure in a simplified setting, for example, in a water tank, are rarely applicable to clinical images, due to the effects of tissue-dependent components.
Therefore, distortions must be estimated based on the backscattered RF data that lead to the B-mode image.
Assuming a narrow ultrasound beam, linear propagation and weak scattering, the ultrasonic RF echo, yi(n), corresponding to the ith axial line in the B-mode image is modeled as [40]:
yi(n)=hi(n)∗fi(n)+wi(n), i=1, 2,. . . (6.44) where n is the discrete time; wi(n)is observation noise; fi(n)represents the underlying tissue structure and is referred to as tissue response; and hi(n)represents the axial distortion kernel. Let us assume that:
(A1) yi(n)is non-Gaussian; (A2) fi(n)is white, non-Gaussian random process; (A3) wi(n)is Gaussian noise uncorrelated with fi(n); and (A4) yi(n)is a short enough segment, so that the attenuation effects stay constant over its duration. (Long segments of data can be analyzed by breaking them into a sequence of short segments.) A similar model can be assumed to hold in the lateral direction of the RF image.
Even though the distortion kernels were known to be non-minimum phase [57], their estimation was mostly carried out using second-order statistics (SOS), such as autocorrelation or power spectrum, thereby neglecting Fourier phase information. Phase is important in preserving edges and boundaries in images.
It is particularly important in medical ultrasound images where the nature of edges of a tumor provide important diagnostic information about the malignancy. Complex cepstrum-based operations that take phase information into account have been used [42] to estimate distortions. HOS retain phase and in addition, are not as sensitive to singularities as the cepstrum. HOS were used [40] for the first time to estimate imaging distortions from B-scan images. It was demonstrated that the HOS-based distortion estimation and subsequent image deconvolution significantly improved resolution. For the case of breast data, in was demonstrated [41] that deconvolution via SOS-based distortion estimates was not as good as its HOS counterpart.
In the following we present some results of distortion estimation followed by deconvolution of clinical B-scan images of human breast data. The data were obtained using a flat linear array transducer with a nominal center frequency of 7.5 MHz on a clinical imaging system UltraMark-9 Advanced Technology Laboratories. Data were sampled at a rate of 20 MHz. Figure 6.3a shows parts of the original image, where the the logarithm of the envelope has been used for display purposes. Axial and lateral distortion kernels were estimated from the RF data via the HOS-based non-parametric method outlined in Section 6.4 of this chapter [21], and also via an SOS power cepstrum based method [41]. Each kernel estimate was obtained from a rectangular block of RF data described by(x, y, Nx, Ny), where (x, y) are the co- ordinates of the upper left corner of the block, Nx is its lateral width, and Nyis the axial height. Note that y corresponds to the depth of the upper left corner of the image block from the surface of the transducer. In the following, all dimensions are specified in sample numbers. The size of the images used was 192×1024. Assuming that within the same block the axial distortion kernel does not significantly depend on the lateral location of the RF data, all axial RF data in the block can be concatenated to form a longer one-dimensional vector. In both HOS and SOS based axial kernel estimations, it was assumed that Nx=10, Ny=128, N =128. Note that the Ny =128 samples correspond to 5 mm in actual tissue space, as is reasonable to assume that attenuation over such a small distance may be assumed constant.
Fifty kernels were estimated from the blocks(x, 400, 10, 128), the x taking values in the range 1,. . ., 50.
Lateral kernels were estimated from the same images with parameters Nx =192, Ny=10, and N =64.
All the kernels were estimated from the blocks(1, y, 192, 10), the y taking values in the range 400,. . ., 600 increasing each time by 5. Data from all adjacent lateral image lines in the block were concatenated to
(a) (b) (c)
Original Deconvolved (HOS) Deconvolved (SOS)
(d) 1 (e) (f)
0.5 0 10
–10 –10 0 10 0
1 0.5 0 10
–10 –10 0 10 0
1 0.5 0 10
–10 –10 0 10 0
FIGURE 6.3 (a) Original image, (b) HOS-based deconvolution, (c) SOS-based deconvolution. In all cases the logarithm of the envelope is displayed. Autocovariance plots for (d) original, (e) HOS deconvolved, and (f) SOS deconvolved RF data.
make a long one-dimensional vector. Note that Ny =10 corresponds to a very small depth, 0.4 mm in the real tissue space, hence he underlying assumption that the lateral distortion kernel does not vary much over this depth, should be reasonable.
Figure 6.3b,c show that the result of lateral followed by axial deconvolution of the image of Figure 6.3a, using respectively HOS- and SOS-based distortion kernel estimates. The deconvolution was performed via the constrained Wiener Filter technique [58]. According to Figure 6.3, the deconvolution in the RF-domain resulted in a significant reduction in speckle size. The speckle size appears smaller in the HOS-deconvolved image than in SOS-deconvolved ones, which is evidence of higher resolution. The resolution improvement can be quantified based on the width of the main lobe of the autocovariance of the image. The axial resolution gain for the HOS-based approach was 1.8 times that of the SOS- based approach. The lateral resolution gain for the HOS-based approach was 1.73 times as much. From a radiologist’s point of view, overall, the HOS image appears to have better spatial resolution than the original as well as the SOS deconvolved images. Conversely, the SOS image seems to have incurred a loss of both axial and lateral resolution.
Acknowledgments
Parts of this chapter have been based on the book: Nikias, C.L. and Petropulu, A.P., Higher Order Spectra Analysis: A Nonlinear Signal Processing Framework, Prentice Hall, Englewood Cliffs, NJ, 1993.
Major support for this work came from the National Science Foundation under grant MIP-9553227, the Whitaker Foundation, the National Institute of Health under grant 2P01CA52823-07A1, Drexel University and the Laboratoire des Signaux et Systems, CNRS, Universite Paris Sud, Ecole Superieure d’Electric, France.
The author would like to thank Drs. F. Forsberg and E. Conant for providing the ultrasound image and for evaluating the processed image.
References
[1] Marple, Jr., S.L. Digital Spectral Analysis with Applications, Prentice-Hall, Inc., Englewood Cliffs, NJ, 1987.
[2] Kay, S.M. Modern Spectral Estimation, Prentice-Hall, Inc., Englewood Cliffs, NJ, 1988.
[3] Haykin, S. Nonlinear Methods of Spectral Analysis, 2nd ed., Springer-Verlag, Berlin, Germany, 1983.
[4] Nikias, C.L. and Petropulu, A.P. Higher-Order Spectra Analysis: A Nonlinear Signal Processing Framework, Prentice Hall Inc., Englewood Cliffs, NJ, 1993.
[5] Schetzen, M. The Volterra and Wiener Theories on Nonlinear Systems, updated edition, Krieger Publishing Company, Malabar, FL, 1989.
[6] Mendel, J.M. “Tutorial on Higher-Order Statistics (Spectra) in Signal Processing and System Theory:
Theoretical Results and Some Applications,” IEEE Proc., 79: 278–305, 1991.
[7] Papoulis, A. Probability Random Variables and Stochastic Processes, McGraw-Hill, New York, 1984.
[8] Nikias, C.L. and Raghuveer, M.R. “Bispectrum Estimation: A Digital Signal Processing Framework,”
Proc. IEEE, 75: 869–891, 1987.
[9] Subba Rao, T. and Gabr, M.M. “An Introduction to Bispectral Analysis and Bilinear Time Series Models,” Lecture Notes in Statistics, 24, New York: Springer-Verlag, 1984.
[10] Brillinger, D.R. and Rosenblatt, M. “Computation and Interpretation of kth-order Spectra,” In:
Spectral Analysis of Time Series, B Harris, Ed., John Wiley & Sons, New York, NY, pp. 189–232, 1967.
[11] Matsuoka, T. and Ulrych, T.J. “Phase Estimation Using Bispectrum,” Proc. IEEE, 72: 1403–1411, October, 1984.
[12] Petropulu, A.P. and Abeyratne, U.R. “System Reconstruction from Higher-Order Spectra Slices,”
IEEE Trans. Sig. Proc., 45: 2241–2251, 1997.
[13] Petropulu, A.P. and Pozidiz, H. “Phase Estimation from Bispectrum Slices,” IEEE Trans. Sig., Proc., 46: 527–531, 1998.
[14] Bartlet, H., Lohmann, A.W., and Wirnitzer, B. “Phase and Amplitude Recovery from Bispectra,”
Applied Optics, 23: 3121–3129, 1984.
[15] Petropulu, A.P. and Nikias, C.L. “Signal Reconstruction from the Phase of the Bispectrum,” IEEE Trans. Acoust., Speech, Signal Processing, 40: 601–610, 1992.
[16] Rangoussi, M. and Giannakis, G.B. “FIR Modeling Using Log-Bispectra: Weighted Least-Squares Algorithms and Performance Analysis,” IEEE Trans. Circuits and Systems, 38: 281–296, 1991.
[17] Pan, R. and Nikias, C.L. “The Complex Cepstrum of Higher-Order Cumulants and Nonminimum Phase System Identification,” IEEE Trans. Acoust., Speech, Signal Process., 36: 186–205, 1988.
[18] Le Roux, J. and Sole, P. “Least-Squared Error Reconstruction of a Deterministic Sampled Signal Fourier Transform Logarithm from its N th Order Polyspectrum Logarithm,” Signal Process., 35:
75–81, 1994.
[19] Lii, K.S. and Rosenblatt, M. “Deconvolution and Estimation of Transfer Function Phase and Coefficients for Nongaussian Linear Processes,” Ann. Stat., 10: 1195–1208, 1982.
[20] Dianat, S.A. and Raghuveer, M.R. “Fast Algorithms for Phase and Magnitude Reconstruction from Bispectra,” Optical Engineering, 29: 504–512, 1990.
[21] Pozidis, H. and Petropulu, A.P. “System Reconstruction from Selected Regions of the Discretized Higher-Order Spectrum,” IEEE Trans. Sig. Proc., 46: 3360–3377, 1998.
[22] Andre, T.F., Nowak, R.D., and Van Veen, B.D. “Low Rank Estimation of Higher Order Statistics,”
IEEE Trans. Sig. Proc., 45: 673–685, March 1997.
[23] Bradaric, I. and Petropulu, A.P. “Low Rank Approach in System Identification Using Higher Order Statistics,” 9th IEEE Sig. Processing Workshop on Statistical Signal and Array Processing — SSAP’98, Portland, Oregon, September 1998.
[24] Bradaric, I. and Petropulu, A.P. “Subspace Design of Low Rank Estimators for Higher Order Statistics,” IEEE Trans. Sig. Proc., submitted in 1999.
[25] Giannakis, G.B. and Mendel, J.M. “Cumulant-Based Order Determination of Non-Gaussian ARMA Models,” IEEE Trans. Acous., Speech, and Sig. Proc., 38: 1411–1423, 1990.
[26] Giannakis, G.B. “Cumulants: A Powerful Tool in Signal Processing,” Proc. IEEE, 75, 1987.
[27] Alshbelli, S.A., Venetsanopoulos, A.N., and Cetin, A.E. “Cumulant Based Identification Approaches for Nonminimum Phase Systems,” IEEE Trans. Sig. Proc., 41: 1576–1588, 1993.
[28] Fonollosa, J.A.R. and Vidal, J. “System Identification Using a Linear Combination of Cumulant Slices,” IEEE Trans. Sig. Process., 41: 2405–2411, 1993.
[29] Petropulu, A.P. “Noncausal Nonminimum Phase ARMA Modeling of Non-Gaussian Processes,”
IEEE Trans. Sig. Process., 43: 1946–1954, 1995.
[30] Tugnait, J. “Fitting Non-Causal AR Signal Plus Noise Models to Noisy Non-Gaussian Linear Processes,” IEEE Trans. Automat. Control, 32: 547–552, 1987.
[31] Tugnait, J. “Identification of Linear Stochastic Systems via Second- and Fourth-Order Cumulant Matching,” IEEE Trans. Inform. Theory, 33: 393–407, 1987.
[32] Tick, L.J. “The Estimation of Transfer Functions of Quadratic Systems,” Technometrics, 3: 562–567, 1961.
[33] Hinich, M.J. “Identification of the Coefficients in a Nonlinear Time Series of the Quadratic Type,”
J. Econom., 30: 269–288, 1985.
[34] Powers, E.J., Ritz, C.K., et al. “Applications of Digital Polyspectral Analysis to Nonlinear Systems Modeling and Nonlinear Wave Phenomena,” Workshop on Higher-Order Spectral Analysis, pp. 73–77, Vail, CO, 1989.
[35] Brillinger, D.R. “The Identification of a Particular Nonlinear Time Series System,” Biometrika, 64:
509–515, 1977.
[36] Rozzario, N. and Papoulis, A. “The Identification of Certain Nonlinear Systems by Only Observing the Output,” Workshop on Higher-Order Spectral Analysis, pp. 73–77, Vail, CO, 1989.
[37] Kim, Y.C. and Powers, E.J. “Digital Bispectral Analysis of Self-Excited Fluctuation Spectra,” Phys.
Fluids, 21: 1452–1453, 1978.
[38] Raghuveer, M.R. and Nikias, C.L. “Bispectrum Estimation: A Parametric Approach,” IEEE Trans.
Acous., Speech, and Sig. Proc., ASSP, 33: 1213–1230, 1985.
[39] Raghuveer, M.R. and Nikias, C.L. “Bispectrum Estimation via AR Modeling,” Sig. Proc., 10: 35–45, 1986.
[40] Abeyratne, U., Petropulu, A.P., and Reid, J.M. “Higher-Order Spectra Based Deconvolution of Ultrasound Images,” IEEE Trans. Ultrason. Ferroelec., Freq. Con., 42: 1064–1075, 1995.
[41] Abeyratne, U.R., Petropulu, A.P., Golas, T., Reid, J.M., Forsberg, F., and Consant, E. “Blind Decon- volution of Ultrasound Breast Images: A Comparative Study of Autocorrelation Based Versus Higher-Order Statistics Based Methods,” IEEE Trans. Ultrason. Ferroelec., Freq. Con., 44: 1409–1416, 1997.
[42] Taxt, T. “Comparison of Cepstrum-Based Methods for Radial Blind Deconvolution of Ultrasound Images,” IEEE Trans. Ultrason., Ferroelec., and Freq. Con., 44, 1997.
[43] Abeyratne, U.R., Petropulu, A.P., and Reid, J.M.“On Modeling the Tissue Response from Ultrasound Images,” IEEE Trans. Med. Imag., 14: 479–490, 1996.
[44] Gurcan, M.N., Yardimci, Y., Cetin, A.E., and Ansari, R. “Detection of Microcalcifications in Mammograms Using Higher-Order Statistics,” IEEE Sig. Proc. Lett., 4, 1997.
[45] Hadjileontiadis, L. and Panas, S.M. “Adaptive Reduction of Heart Sounds from Lung Sounds Using Fourth-Order Statistics,” IEEE Trans. Biomed. Eng., 44, 1997.
[46] Upshaw, B. “SVD and Higher-Order Statistical Processing of Human Nerve Signals,” Proc.
International Conference of the IEEE Engineering in Medicine and Biology, 1996.
[47] Ning, T. and Bronzino, J.D. “Bispectral Analysis of the EEG During Various Vigilance States,” IEEE Trans. Biomed. Eng., 36: 497–499, 1989.
[48] Samar, V.J., Swartz, K.P., et al. “Quadratic Phase Coupling in Auditory Evoked Potentials from Healthy Old Subjects and Subjects with Alzheimer’s Dementia,” IEEE Signal Processing Workshop on Higher-Order Statistics, pp. 361–365, Tahoe, CA, 1993.
[49] Tang, Y. and Norcia, A.M. “Coherent Bispectral Analysis of the Steady-State VEP,” Proc. International Conference of the IEEE Engineering in Medicine and Biology, 1995.
[50] Husar, P. and Henning, G. “Bispectrum Analysis of Visually Evoked Potentials,” IEEE Eng. Med.
Biol., 16, 1997.
[51] Henning, G. and Husar, P. “Statistical Detection of Visually Evoked Potentials,” IEEE Eng. Med. Biol., 14: 386–390, 1995.
[52] “Bispectral Analysis of Visual Field Interactions,” Proc. International Conference of the IEEE Engineering in Medicine and Biology, vol. 3, Amsterdam, Netherlands, 1996.
[53] Yana, K., Mizuta, H., and Kajiyama, R. “Surface Electromyogram Recruitment Analysis Using Higher-Order Spectrum,” Proc. International Conference of the IEEE Engineering in Medicine and Biology, 1995.
[54] Orizio, C., Liberati, D., Locatelli, C., De Grandis, D., Veicsteinas, A., “Surface Mechanomyogram Reflects Muscle Fibres Twitches Summation,” J. Biomech., 29, 1996.
[55] Barnett, T.P., Johnson, L.C., et al., “Bispectrum Analysis of Electroencephalogram Signals During Waking and Sleeping,” Science, 172: 401–402, 1971.
[56] Husar, P.J., Leiner, B., et al., “Statistical Methods for Investigating Phase Relations in Stochastic Processes,” IEEE Trans. Audio and Electroacoust., 19: 78–86, 1971.
[57] Jensen, J.A. and Leeman, S. “Nonparametric Estimation of Ultrasound Pulses,” IEEE Trans. Biomed.
Eng., 41: 929–936, 1994.
[58] Treitel, S. and Lines, L.R. “Linear Inverse Theory and Deconvolution,” Geophysics, 47: 1153–1159, 1982.
[59] Cohen, F.N. “Modeling of Ultrasound Speckle with Applications in Flaw Detection in Metals,” IEEE Trans. Sig. Proc., 40: 624–632, 1992.
[60] Giannakis, G.B. and Swami, A. “On Estimating Noncausal Nonminimum Phase ARMA Models of Non-Gaussian Processes,” IEEE Tran. Acous., Speech, Sig. Process., 38: 478–495, 1990.
[61] Jensen, J.A. “Deconvolution of Ultrasound Images,” Ultrasonic Imaging, 14: 1–15, 1992.
[62] Le Roux, J, Coroyer, C., and Rossille, D. “Illustration of the Effects of Sampling on Higher-Order Spectra,” Sig. Process., 36: 375–390, 1994.
[63] Ning, T. and Bronzino, J.D. “Autogressive and Bispectral Analysis Techniques: EEG Applications,”
IEEE Engineering in Medicine and Biology, March 1990.
[64] Oppenheim, A.V. and Schafer, R.W. Discrete-Time Signal Processing, Prentice-Hall, Englewood Cliffs, NJ, 1989.
[65] Petropulu, A.P. “Higher-Order Spectra in Signal Processing,” Signal Processing Handbook, CRC Press, Boca Raton, FL, 1998.
[66] Swami, A. and Mendel, J.M. “ARMA Parameter Estimation Using Only Output Cumulants,” IEEE Trans. Acoust., Speech and Sig. Process., 38: 1257–1265, 1990.
7
Neural Networks in Biomedical Signal Processing
Evangelia
Micheli-Tzanakou
Rutgers University
7.1 Neural Networks in Sensory Waveform Analysis . . . 7-2 Multineuronal Activity Analysis •Visual Evoked Potentials
7.2 Neural Networks in Speech Recognition . . . 7-5 7.3 Neural Networks in Cardiology . . . 7-7 7.4 Neural Networks in Neurology . . . 7-11 7.5 Discussion . . . 7-11 References . . . 7-11 Computing with neural networks (NNs) is one of the faster growing fields in the history of artificial intelligence (AI), largely because NNs can be trained to identify nonlinear patterns between input and output values and can solve complex problems much faster than digital computers. Owing to their wide range of applicability and their ability to learn complex and nonlinear relationships — including noisy or less precise information — NNs are very well suited to solving problems in biomedical engineering and, in particular, in analyzing biomedical signals.
Neural networks have made strong advances in continuous speech recognition and synthesis, pattern recognition, classification of noisy data, nonlinear feature detection, and other fields. By their nature, NNs are capable of high-speed parallel signal processing in real time. They have an advantage over conventional technologies because they can solve problems that are too complex — problems that do not have an algorithmic solution or for which an algorithmic solution is too complex to be found. NNs are trained by example instead of rules and are automated. When used in medical diagnosis, they are not affected by factors such as human fatigue, emotional states, and habituation. They are capable of rapid identification, analysis of conditions, and diagnosis in real time.
The most widely used architecture of an NN is that of a multilayer perceptron (MLP) trained by an algorithm called backpropagation (BP). Backpropagation is a gradient-descent algorithm that tries to minimize the average squared error of the network. In real applications, the network is not a simple one-dimensional system, and the error curve is not a smooth, bowl-shaped curve. Instead, it is a highly complex, multidimensional curve with hills and valleys (for a mathematical description of the algorithm).
BP was first developed by Werbos in 1974 [1], rediscovered by Parker in 1982 [2], and popularized later by Rummelhart et al. in 1986 [3]. There exist many variations of this algorithm, especially trying
7-1
to improve its speed and performance in avoiding getting stuck into local minima — one of its main drawbacks.
In my work, I use the ALOPEX algorithm developed by my colleagues and myself (see Chapter 182) [4–10], and my colleagues and I have applied it in a variety of world problems of considerable complexity.
This chapter will examine several applications of NNs in biomedical signal processing. One- and two- dimensional signals are examined.