Image Processing The Fundamentals Image Processing The Fundamentals Maria Petrou University of SurreN Guildford, UK Panagiota Bosdogianni Technical Universify of Crete, Chania, Greece JOHN WILEY & SONS, LTD Chichester - New York WeinheimBrisbaneSingaporeToronto Copyright 1999 by John Wiley & Sons Ltd, Baffis Lane, Chichester, West SussexP019 IUD, England National 01243 779711 International (+44) 1243 779177 e-mail (for orders and customer service enquiries): cs-books@wiley.co.uk Visit ourHome Page on http://www.wiley.co.uk or http://www.wiley.com All Rights Reserved No partof this publication may be reproduced, stored in a retrieval system, or transmitted, in any form orby any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the of a licence issued by the Copyright terms of the Copyright, Designs and Patents Act 1988 or under the terms Licensing Agency, 90 Tottenham Court Road, London, UK W1P 9HE, without the permission in writing of the Publisher, with the exception any material supplied specifically for the purpose of being entered and executed on of a computer system, for exclusive by the purchaserof the publication use Designations used by companies distinguish their products are often claimed to as trademarks In all instances where John Wiley Sons is aware of a claim, product name appear in initial capital or all capital letters & the Readers, however, should contact appropriate companies for more complete information regarding trademarks the and registration Other Wiley Editorial Ofices John Wiley & Sons, Inc., 605 Third Avenue, New York, NY10158-0012,USA WLEY-VCH Verlag GmbH, Pappelallee 3, D69469 Weinheim, Germany Jacaranda Wiley Ltd, Park Road, Milton, 33 Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, ClementiLoop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons (Canada) Ltd, 22 Worcester Road Rexdale, Ontario, M9W 1L1, Canada Library o Congress Cataloging-in-Publication f Data Petrou,Maria Image Processing : the fundamentals / Maria Petrou, Panagiota Bosdogianni cm p Includes bibliographical references ISBN 0-471-99883-4 (alk paper) I Image processing-Digital techniques Bosdogianni, Panagiota II Title TA1637.P48 1999 621.36 ' ~ 99-32327 CIP British Library Cataloguing Publication Data in A catalogue record for book is available from the British Library this ISBN 0-471-99883-4 Produced from PostScript files supplied the authors by Printed and bound in Great Britain Bookcraft (Bath) Limited by This book is printed onacid-free paper responsibly manufactured from sustainable forestry, in which two at least trees are planted for each one used for paper production To the Costases in our lives tions Contents Preface xv List of Figures xvii Introduction Why we process images? What is an image? What is the brightness of an image at a pixel position? Why are images often quoted as being 512 X 512 256 X 256 128 X 128 etc? How many bits we need to store an image? What is meant by image resolution? How we Image Processing? What is a linear operator? How are operators defined? How does an operator transform an image? What is the meaning of the point spread function? How can we express in practice the effect of a linear operator on an image? What is the implication of the separability assumption on the structure of matrix H? How can a separable transform be written in matrix form? What is the meaning of the separability assumption? What is the “take home” message of this chapter? What is the purpose of Image Processing? What is this book about? Image What is thischapterabout? How can we define an elementaryimage? What is the outer product of two vectors? How can we expand an image in terms of vector outer products? What is a unitarytransform? What is a unitarymatrix? What is the inverse of a unitary transform? How can we construct a unitary matrix? vii 1 2 2 6 6 14 15 15 18 18 18 21 21 21 21 23 23 24 24 viii Image Processing: The Fundamentals How should we choose matrices U and V so that g can be represented by fewer bits than f ? 24 24 How can we diagonalize a matrix? How can we compute matrices U V and A + needed for the image diagonalization? 30 What is the singular value decomposition of an image? 34 How can we approximate an image using SVD? 34 What is the error of the approximation of an image by SVD? 35 How can we minimize the error of the reconstruction? 36 What are the elementary images in terms of which SVD expands an image? 37 Are there any sets of elementary images in terms of which ANY image can beexpanded? 45 What is a complete and orthonormal set of functions? 45 Are there any complete sets of orthonormal discrete valued functions? 46 How are the Haar functions defined? 46 How are the Walsh functions defined? 47 How can we create the image transformation matrices from the Haar and Walsh functions? 47 What the elementary images of the Haar transform look like? 51 Can we define an orthogonal matrix with entries only +l or - l ? 57 What the basis images of the Hadamard/Walsh transform look like? 57 What are the advantages and disadvantages of the Walsh and the Haar transforms? 62 What is the Haar wavelet? 62 What is the discrete version of the Fourier transform? 63 How can we write the discrete Fourier transform in matrix form? 65 Is matrix U used for DFT unitary? 66 Which are the elementary images in terms of which DFT expands an image? 68 Why is the discrete Fourier transform more commonly used than the other transforms? 72 What does the convolution theorem state? 72 How can we display the discrete Fourier transform of an image? 79 What happens to the discrete Fourier transform of an image if the image is rotated? 79 What happens to the discrete Fourier transform of an image if the image is shifted? 81 What is the relationship between the average value of a function and its DFT? 82 What happens to the DFT of an image if the image is scaled? 83 What is the discrete cosine transform? 86 What is the “take home” message of this chapter? 86 Statistical Description of Images What is this chapter about? Why we need the statistical description of images? 89 89 89 t Contents Is there an image transformation that allows its representation in terms of uncorrelated data that can be used to approximate the image in the least mean square error sense? What is a random field? What is a random variable? How we describerandom variables? What is the probability of an event? What is the distribution function of a random variable? What is the probability of a random variable taking a specific value? What is the probability density function of a random variable? How we describemanyrandom variables? What relationships may n random variables have with each other? How we then define a random field? How can we relate two random variables that appear in the same random field? How can we relate two random variablesthat belong to two different random fields? Since we always have just one version of an image how we calculate the expectation values that appear in all previous definitions? When is a random fieldhomogeneous? How can we calculate the spatial statistics of a random field? When is a random field ergodic? When is a random field ergodic with respect to the mean? When is a random field ergodic with respect to the autocorrelation function? What is the implication of ergodicity? How can we exploit ergodicity to reduce the number of bits needed for representing an image? What is the form of the autocorrelation function of a random field with uncorrelatedrandom variables? How can we transform the image so that its autocorrelation matrix is diagonal? Is the assumption of ergodicity realistic? How can we approximate an image using its K-L transform? What is the error with which we approximate an image when we truncate its K-L expansion? What are the basis images in termsof which the Karhunen-Loeve transform expands an image? What is the “take home” message of this chapter? Image What is imageenhancement? How can we enhance an image? Which methods of the image enhancement reason about the grey level statistics of an image? ix 89 90 90 90 90 90 91 91 92 92 93 94 95 96 96 97 97 97 97 102 102 103 103 104 110 110 111 124 125 125 125 125 Image Processing: The Fundamentals 320 The right hand side of this expression is a random variable, indicating that the location of the edge will be marked at various randomly distributed positions around the true position, which is at zo = We can calculate the mean shifting away from the true position as the expectation value of 20 This, however, is expected to be So, we calculateinsteadthe variance of the zo values We square both sides of (7.45) and take the expectationvalue of Note that the expectationvalue operator applies only to the random components In Box B7.3 we saw that if a noise signal with variance ni is convolved by a filter f ( z ) ,the mean square value of the output signal is given by: 1cc ni f"4dX (see equation (7.37)) The right hand side of equation (7.45) here indicates the convolution of the noise component by filter f'(z) Its mean square value therefore is ni S-", [ f ' ( ~ ) ] ~ d z Then: (7.46) The smaller this expectation value is, the better the localization of the edge We can define, therefore, the good locality measure as the inverse of the square root of the above quantity We may also ignore factor no as it is the standard deviation of the noise during the imaging process, over which we not have control So a filter is optimal with respect t o good locality, if it maximizes the quantity: (7.47) I I Example 7.17 (B) Show that the differentiation of the output of a convolution of a signal with a filter f ( x ) can be achieved by convolving the signal with the derivative of the filter U(%) Image Segmentation and Edge Detection 321 The output of the convolution is: or (7.48) Applying Leibnitz's rule for differentiating an integral with respect to a parameter (Box B7.1) we obtain from equation (7.48): which upon changing variables can be written as: B7.6: Derivation of the count of false maxima It has been shown by Rice that the average density of zero crossings of the convolution of a function h with Gaussian noise is given by: "' where Rhh(7) [ -&h@) R"hh (0) ] (7.49) is the spatial autocorrelation function of function h ( z ) ,i.e: (7.50) Therefore: Using Leibnitz's rule (see r to obtain: Box B7.1) we can differentiate (7.50) with respect, t o 322 Image Processing: The Fundamentals Rthh(7) = J' 0 h(X)h'(X + 7)dX -cc We define a new variable of integration obtain: R$&) = J' = X +r + X = - r and dx = d5 to 0 h(5 - r)ht(5)d5 (7.51) -cc We differentiate (7.51) once more: 0 R'thh(7) = [ J-, 0 + R'lhh(0) = [ ht(5 - T)ht(5)d5 - (h'(x))2dx J -cc Therefore, the average distance of zero crossings of the output signal when a noise signal is filtered by function h(x) is given by: (7.52) From the analysis in Box B7.5, can see that the false maxima in our casewill we come from equation S-, f'(x)n(zo- z)dz = which will give the false alarms " in the absence of any signal (U(.) = 0) This is equivalent to saying that the false maxima coincide with the zeroes in the output signal when the noise signal is filtered with function f'(x) So, ifwe want to reduce the number of false local maxima, we should make the average distance between the zero crossings as large as possible, for the filter function f'(z) Therefore, we define the good measure of scarcity of false alarms as: What is the "take home" message of this chapter? This chapter dealt with the reduction of the information content of an image so that it can be processed more easily by a computer vision system It surveyed the two basic approaches for this purpose: region segmentation and edge detection Region segmentation triesto identify spatially coherent sets of pixels that appear to constitute uniform patches in the image These patches may represent surfaces or parts of surfaces of objects depicted in the image Edge detection seeks to identify boundaries between such uniform patches The most common approach for this is based on the estimate of the first derivative of the image For images with low levels of noise the Image Segmentation and Edge Detection 323 Sobel masks are used to enhance the edges For noisy images the Canny filters should be used Canny filters can be approximated by the derivative of a Gaussian; i.e they have the form x e - Although parameter (T in this expression has to have a specific value for the filters to be optimal according to Canny’s criteria, often people treat (T as a free parameter and experiment with various values of it Care must be taken in this case when discretizing the filter and truncating the Gaussian, not to create a filter with sharp ends Whether the Sobel masks are used or the derivatives of the Gaussian, the result is the enhancement of edges in the image The output produced has to be further processed by non-maxima suppression (i.e the identification of the local maxima in the output array) and thresholding (i.e the retention of only the significant local maxima) Edge detectors consist of all three stages described above and produce fragmented edges Often a further step is involved of linking the fragments together to create closed contours that identify uniform regions Alternatively, people may bypass this process by performing region segmentation directly and, if needed, extract theboundaries of the regions afterwards Region-based methods are much more powerful when both attribute similarity and spatial proximity are taken into consideration when deciding which pixels form which region Bibliography The material of this book is based largely on classical textbooks The method described in example 6.10 was taken fromreference Chapter is based mainly on research papers From the references given below, n u d e r s 13, 15 and 23 refer to thresholding methods References 5, 6, 13, 18-22 and 28 refer to linear edge detectors Paper 17 refers to non-linear edge detectors with an application of them presented in 10 Reference 21 is an extensive review of the Canny-related approaches to edge detection M Abramowitz and I A Stegun (eds), 1970 Handbook of Mathematical Functions, Dover Publications, ISBN 0-486-61272-4 K G Beauchamp, 1975 Walsh Functions and their Applications, Academic Press, ISBN 0-12-084050-2 C R Boukouvalas, 1996 Colour Shade Grading and its Applications to Visual Inspection PhD thesis, University of Surrey, UK R N Bracewell, 1978 The Fourier Transform and its Applications, McGrawHill, ISBN 0-07-007013-X J Canny, 1983 Finding Edges and Lines in Images MIT A1 Lab Technical Report 720 J Canny, 1986 "A computational approach to edge detection" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol P AMI-8, pp 679-698 R A Gabel and R A Roberts, 1987 Signals and Linear Systems, John Wiley & Sons, ISBN 0-471-83821-7 R C Gonzalez and R E Woods, 1992 Digital Image Processing Addison-Wesley , ISBN 0-201-50803-6 I S Gradshteyn and I M Ryzhik, 1980 Table of Integrals, Series and Products, Academic Press, ISBN 0-12-294760-6 10 J Graham and C J Taylor, 1988 "Boundary cue operators for model-based image processing" Proceedings of the Fourth Alvey Vision Conference, AVC88, University of Manchester, 31 August-2 September 1988, pp 59-64 326 Image Processing: The Fundamentals 11 H P Hsu, 1970 Fourier Analysis, Simon & Schuster, New York 12 T S Huang (ed), 1979 “Picture processing and digital filtering” In Applied Physics, Vol 6, Springer-Verlag, ISBN 0-387-09339-7 Topics in 13 J Kittler, 1983 “On the accuracy of the Sobel edge detector” Imageand Vision Computing, Vol 1, pp 37-42 14 J Kittlerand J Illingworth, 1985 “Onthreshold selection using clustering criteria” IEEE Transactions on Systems, Man, and Cybernetics, Vol SMC-15, pp 652-655 15 N Otsu, 1979 “A threshold selection method from gray level histograms” IEEE Transactions on Systems, Man, and Cybernetics, Vol SMC-9, pp 62-66 Random Variables Stochastic and Processes, 16 A Papoulis, 1965 Probability, McGraw-Hill Kogakusha Ltd, Library of Congress Catalog Card number 6422956 17 I Pitas and A N Venetsanopoulos, 1986 “Non-linear order statistic filters for image filtering and edge detection” Signal Processing, Vol 10, pp 395-413 18 M Petrou and J Kittler, 1991 “Optimal edge detectors for ramp edges” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol PAMI-13, pp 483-491 19 M Petrou and A Kolomvas, 1992 “The recursive implementation of the optimal filter for the detection of roof edges and thin lines” In Signal Processing VI? Theory and Applications, pp 1489-1492 20 M Petrou, 1993 “Optimal convolution filters and an algorithm for the detection of wide linear features.” IEE Proceedings: Vision, Signal and Image Processing, V01 140, pp 331-339 21 M Petrou, 1994 “The differentiating filter approach to edge detection” Advances in Electronics and Electron Physics, Vol 88, pp 297-345 22 M Petrou, 1995 “Separable 2D filters for the detection of ramp edges” IEE Proceedings: Vision, Image and Signal Processing, Vol 142, pp 228-231 23 M Petrou and A Matrucceli, 1998 “On the stability of thresholding SAR images” Pattern Recognition, Vol 31, pp 1791-1796 24 W K Pratt, 1978 Digital Image Processing, John Wiley & Sons, ISBN 0-47101888-0 25 R M Pringle and A A Rayner, 1971 Generalized Inverse Matrices with Applications to Statistics, No 28 of Griffin’s Statistical monographs and courses, edited by A Stuart, ISBN 0-85264-181-8 Bibliography 327 26 S Rice, 1945 “Mathematical analysis of random noise” Bell Systems Tech J., V01 24, pp 46-156 27 A Rosenfeld and A C Kak, 1982 Digital Picture Processing, Academic Press, ISBN 0-12-597301-2 28 L A Spacek, 1986 “Edge detection and motion detection” Image and Vision Computing, Vol4, pp 43-53 29 P H Winston, 1992 Artificial Intelligence, Addison-Wesley, ISBN 0-20-1533774 30 T Y Young and K S Fu (eds), 1986 Handbook of Pattern Recognition and Image Processing, Academic Press, ISBN 0-12-774560-2 Index A C additive noise 144 antisymmetric image features 309 approximation theory for filter design 171-191 approximation of an image using its K-L transform 110, 115 approximation of an image by SVD 34,35 autocorrelation function of a random field 9.4, 103 autocorrelation matrix of an image (how to compute) 98-102, 105-106 autocovariance of a random field 94 automatic vision 18 Canny’s criteria for an optimal edge filter 306-309,318-322 characteristic equation of a matrix 31 Chebyshev norm 171, 172 checkerboard effect circulant matrix 233 clustering 288 complete set of functions 45 conditional expectation 217 constraint matrix inversion 251 contrast enhancement of a multispectral image 135 contrast manipulation 132 convolution 7, 72-77, 157 convolution theorem (assumptions of) 75 covariance of two random variables 92 cross correlation of two random fields 95 cross covariance of two random fields 95 cross spectral density of two random fields 218 B bandpass filter 160 bands of an image 1, 138 barrel distortion 194 basis images forthe Fourier transform 68, 69 basis images for the Haar transform 52 basis images for the Hadamard transform 58 basis images from the K-L transform 111 basis images for the Walsh transform 58 bilinear interpolation 195 bits needed for an image block circulant matrix 232, 239-240 brightness of an image pixel D delta function DFT (see discrete Fourier transform) direct component of an image 82 discrete cosine transform 86 discrete Fourier transform 63-86 discrete Fourier transform (display of) 79 discrete Fourier transform and image restoration 210-215, 228-230 discrete Fourier transform of a rotated image 79 discrete Fourier transform of a scaled 330 image 83 discrete Fourier transform of a shifted image 81 distribution function of a random variable 90 distribution function of many random variables 92 dual grid 291 E edge detection 265, 289-322 edge map 303 edge pixels 291 edgels 291 eigenimages from the K-L transform 121 eigenimages from SVD 37 eigenvalues of a circulant matrix 233 eigenvalues of a matrix 30, 140, 234 eigenvectors of a circulant matrix 233 eigenvectors of a matrix 30, 31, 140, 234 ergodic random field 96, 97 ergodic random field with respect to the mean 97 ergodic random field with respect to the autocorrelation function 97 ergodicity lU2, 103, 104, 227 error in approximating an image by its K-L transform 115 error in approximating an image by its SVD 35 even symmetrical cosine transform 86 expansion of an image in terms of Fourier basis functions 68 expansion of an image in terms of eigenimages 34 expansion of an image in terms of Haar basis functions 51-52, 58-60 expansion of an image in terms of Walsh/ Hadamard matrices 57-58, 60-61 expansion of an image using its K-L transform 110 expansion of an image in terms of vector outer products 21-22 Image Processing: The Fundamentals expected value of a random variable 92 F false contouring fast Fourier transform 84 feature map 315-316 features 288 FFT (see fast Fourier transform) filter 155 Fourier transform 45, 63, 72 frequency convolution theorem 75 frequency sampling 172, 182-191 Fresnel integrals 202-204 G Gaussian noise 144 geometric progression 64 geometric restoration 193-198 grey level grey level interpolation 195 H Haar functions 46 Haar transforms 47, 62 Haar wavelet 62 Hadamard matrices 57 Hadamard transforms 57 highpass filter 148, 160 histogram equalization 127 histogram hyperbolization 129 histogram modification with random additions 127, 130 histogram of an image 125 histogram of an image under variable illumination 283-285 homogeneous random field 96 homomorphic filter 149-150 Hotelling transform 89 hysteresis thresholding 266 Index 331 I K ideal highpass filter 148, 160 ideal lowpass filter 148, 157-161 image image as a linear superposition of point sources image as a random field 89-121 image classification 266 image compression 18 image compression using K-L 102 image compression using SVD 24 image enhancement 18, 125-153 image geometric restoration 193-198 image labelling 266 image registration 193 image resolution image restoration 18, 193-263 image restoration by inverse filtering 209-217 image restoration by matrix inversion 230-262 image restoration by Wiener filtering 218-230 image segmentation 265 image sharpening 148 image smoothing 147 image thresholding 255-286 image thresholding under variable illumination 285 impulse noise 144 independent random variables 92 inhomogeneous contrast 131 Karhunen-Loeve expansion of an image 110 Karhunen-Loeve transform 89 Karhunen-Loeve transform of a multispectral image 136 K-L (see Karhunen-Loeve) Kronecker ordering of Walsh functions 57 Kronecker product of matrices 14, 237 La Vallee Poussin theorem 180, 181 Lagrange multipliers 251-258 Laplacian 243 least square error solution for image restoration 218 least square error approximation of an image 37 Leibnitz rule for differentiating an integral with respect to a parameter 268 lexicographic ordering of Walsh functions 57 limiting set of equations 180 line detection 309-310 linear operator linear programming 172, 174 local contrast enhancement 131 low pass filter 148, 157, 161 lowpass filtering 147 J M joint distribution function of many random variables 92 joint probability density function of many random variables 92 L matrix diagonilization 24-35 maximizing algorithms for filter design 180 mean square error for K-L transform 118 mean value of a random variable 92 median filter 146 minimizing algorithms for filter design 180 mini-max algorithms for filter design 180 minimum error threshold 268-278 minimum mean square error 332 Processing: Image approximation of an image 118, 124 minimum square error approximation of an image 36 motion blurring 200-204, 210-217, 228-230,259-262 multiband image 135 multiplicative noise 144, 149, 283 multispectral image 135 N natural order of Walsh functions 47 nearest neighbour interpolation 195 noise 144 noise convolved with a filter 316-317 noise in image restoration 210 non-maxima suppression 303 non-recursive filter 161 norm of a matrix 35 operator optimal threshold 268-278 orthogonal matrix 24 orthogonal random variables 92 orthogonal set of functions 45 orthonormal set of functions 45 orthonormal vectors 24 Otsu’s thresholding method 278-282 outer product of vectors 21 P partition of a matrix 10 pattern recognition 288 pel l pincushion distortion 194 pixel point source 7-9 point spread function of a linear degradation process 198-209 point spread function of an operator 6-7 The Fundamentals principal component analysis of a multispectral image 136-144 probability of an event 90 probability density function of a random variable 91, 92 probability density function of many random variables 92 properties of a discrete Fourier transform 79-84 Q quadtree 289 R ramp edges 309 random field 90, 93 random variable 90 rank order filtering 146 rectangle function recursive filter 161 region growing 288 resolution restoration by matrix inversion 230-262 restoration by Wiener filtering 218-230 restoration of motion blurring 210-217, 228-230, 259-262 Rice’s formula for filtered noise 321 Robinson operators 208 S salt and pepper noise 144 scaling function 62 seed pixels for region growing 288 separability assumption 14-15 separable masks 306 separable point spread function separable transform 15 sequency order of Walsh functions 47 sharpening 148 shift invariant point spread function Index shifting property of the delta function simulated annealing 217 singular value decomposition of an image 34 singular value decomposition of a matrix 24 smoothing 147 Sobel masks 296-299,306 spatial statistics of a random field 97 spatial autocorrelation matrix an image of 105 spectral bands 1, 138 spectral density of a random field 223 split and merge algorithms 288 stability condition in filter design 171 stacking operator 10 standard deviation of a random variable 92 successive doubling algorithm 84 SVD (see singular value decomposition) system function 155 T 333 uncorrelated random variables 92 unit sample response of a filter 155 unitary matrix 23-24 unitary transform 23 V variable illumination 149, 283-286 variance of a random variable 92 vector outer product 21 vector outer product (expansion of an image in terms of) 21-22 W Walsh functions 46, 47 Walsh transforms 47, 62 wavelets 62 white noise 220 Wiener filter 218-230, 254 Wiener-Khinchine theorem 223 windowing 172, 173, 210 textured regions 288 Z thresholding 255 tie points 195 z-transform 161-171 time convolution theorem 75 trace of a matrix 36 transfer function from an astronomical image 204 transfer function from a bright edge 206-209 transfer function from a bright line 205-206 transfer function of a degradation process 199-209 transfer function of motion blurring 200-204,210-217 U uncorrelated data 144 uncorrelated random fields 95 ... Petrou ,Maria Image Processing : the fundamentals / Maria Petrou, Panagiota Bosdogianni cm p Includes bibliographical references ISBN 0-4 7 1-9 988 3-4 (alk paper) I Image processing-Digital techniques Bosdogianni, ... by A-$ to get: A - + S ~ ~ ~ T S T A - +A - ~ A A - + = I T Since A-+ is diagonal, A-+ = (A-?) as : I (2.17) So the above equation can be rewritten T A - + s ~ ~ ( A - + s ~ ~ )I = (2.18) Therefore,... (1.19) is the value of the image g at n: = a , y = b, i.e + -+ + -& <