121 5 DISCRETE IMAGE MATHEMATICAL CHARACTERIZATION Chapter 1 presented a mathematical characterization of continuous image fields. This chapter develops a vector-space algebra formalism for representing discrete image fields from a deterministic and statistical viewpoint. Appendix 1 presents a summary of vector-space algebra concepts. 5.1. VECTOR-SPACE IMAGE REPRESENTATION In Chapter 1 a generalized continuous image function F(x, y, t) was selected to represent the luminance, tristimulus value, or some other appropriate measure of a physical imaging system. Image sampling techniques, discussed in Chapter 4, indicated means by which a discrete array F(j, k) could be extracted from the contin- uous image field at some time instant over some rectangular area , . It is often helpful to regard this sampled image array as a element matrix (5.1-1) for where the indices of the sampled array are reindexed for consistency with standard vector-space notation. Figure 5.1-1 illustrates the geometric relation- ship between the Cartesian coordinate system of a continuous image and its array of samples. Each image sample is called a pixel. J– jJ≤≤ K– kK≤≤ N 1 N 2 × F Fn 1 n 2 ,()[]= 1 n i N i ≤≤ Digital Image Processing: PIKS Inside, Third Edition. William K. Pratt Copyright © 2001 John Wiley & Sons, Inc. ISBNs: 0-471-37407-5 (Hardback); 0-471-22132-5 (Electronic) 122 DISCRETE IMAGE MATHEMATICAL CHARACTERIZATION For purposes of analysis, it is often convenient to convert the image matrix to vector form by column (or row) scanning F, and then stringing the elements together in a long vector (1). An equivalent scanning operation can be expressed in quantita- tive form by the use of a operational vector and a matrix defined as (5.1-2) Then the vector representation of the image matrix F is given by the stacking opera- tion (5.1-3) In essence, the vector extracts the nth column from F and the matrix places this column into the nth segment of the vector f. Thus, f contains the column- FIGURE 5.1-1. Geometric relationship between a continuous image and its array of samples. N 2 1× v n N 1 N 2 ⋅ N 2 × N n v n 0 0 1 0 0 = … … 1 n 1– n n 1+ N 2 … … N n 0 0 1 0 0 = …… 1 n 1– n n 1+ N 2 …… fN n Fv n n 1 = N 2 ∑ = v n N n GENERALIZED TWO-DIMENSIONAL LINEAR OPERATOR 123 scanned elements of F. The inverse relation of casting the vector f into matrix form is obtained from (5.1-4) With the matrix-to-vector operator of Eq. 5.1-3 and the vector-to-matrix operator of Eq. 5.1-4, it is now possible easily to convert between vector and matrix representa- tions of a two-dimensional array. The advantages of dealing with images in vector form are a more compact notation and the ability to apply results derived previously for one-dimensional signal processing applications. It should be recognized that Eqs 5.1-3 and 5.1-4 represent more than a lexicographic ordering between an array and a vector; these equations define mathematical operators that may be manipulated ana- lytically. Numerous examples of the applications of the stacking operators are given in subsequent sections. 5.2. GENERALIZED TWO-DIMENSIONAL LINEAR OPERATOR A large class of image processing operations are linear in nature; an output image field is formed from linear combinations of pixels of an input image field. Such operations include superposition, convolution, unitary transformation, and discrete linear filtering. Consider the element input image array . A generalized linear operation on this image field results in a output image array as defined by (5.2-1) where the operator kernel represents a weighting constant, which, in general, is a function of both input and output image coordinates (1). For the analysis of linear image processing operations, it is convenient to adopt the vector-space formulation developed in Section 5.1. Thus, let the input image array be represented as matrix F or alternatively, as a vector f obtained by column scanning F. Similarly, let the output image array be represented by the matrix P or the column-scanned vector p. For notational simplicity, in the subsequent discussions, the input and output image arrays are assumed to be square and of dimensions and , respectively. Now, let T denote the matrix performing a linear transformation on the input image vector f yielding the output image vector (5.2-2) FN n T fv n T n 1 = N 2 ∑ = N 1 N 2 × Fn 1 n 2 ,() M 1 M 2 × Pm 1 m 2 ,() Pm 1 m 2 ,() Fn 1 n 2 ,()On 1 n 2 m 1 m 2 ,;,() n 2 1 = N 2 ∑ n 1 1 = N 1 ∑ = On 1 n 2 m 1 m 2 ,;,() Fn 1 n 2 ,() Pm 1 m 2 ,() N 1 N 2 N== M 1 M 2 M== M 2 N 2 × N 2 1× M 2 1× pTf= 124 DISCRETE IMAGE MATHEMATICAL CHARACTERIZATION The matrix T may be partitioned into submatrices and written as (5.2-3) From Eq. 5.1-3, it is possible to relate the output image vector p to the input image matrix F by the equation (5.2-4) Furthermore, from Eq. 5.1-4, the output image matrix P is related to the input image vector p by (5.2-5) Combining the above yields the relation between the input and output image matri- ces, (5.2-6) where it is observed that the operators and simply extract the partition from T. Hence (5.2-7) If the linear transformation is separable such that T may be expressed in the direct product form (5.2-8) MN× T mn T T 11 T 12 … …… … T 1N T 21 T 22 … …… … T 2N T M1 T M2 … T MN = … … … pTN n Fv n n 1 = N ∑ = PM m T pu m T m 1 = M ∑ = PM m T TN n ()Fv n u m T () n 1 = N ∑ m 1 = M ∑ = M m N n T mn PT mn Fv n u m T () n 1 = N ∑ m 1 = M ∑ = TT C T R ⊗= GENERALIZED TWO-DIMENSIONAL LINEAR OPERATOR 125 where and are row and column operators on F, then (5.2-9) As a consequence, (5.2-10) Hence the output image matrix P can be produced by sequential row and column operations. In many image processing applications, the linear transformations operator T is highly structured, and computational simplifications are possible. Special cases of interest are listed below and illustrated in Figure 5.2-1 for the case in which the input and output images are of the same dimension, . FIGURE 5.2-1. Structure of linear operator matrices. T R T C T mn T R mn,()T C = PT C F T R mn,()v n u m T n 1 = N ∑ m 1 = M ∑ T C FT R T == MN= 126 DISCRETE IMAGE MATHEMATICAL CHARACTERIZATION 1. Column processing of F: (5.2-11) where is the transformation matrix for the jth column. 2. Identical column processing of F: (5.2-12) 3. Row processing of F: (5.2-13) where is the transformation matrix for the jth row. 4. Identical row processing of F: (5.2-14a) and (5.2-14b) 5. Identical row and identical column processing of F: (5.2-15) The number of computational operations for each of these cases is tabulated in Table 5.2-1. Equation 5.2-10 indicates that separable two-dimensional linear transforms can be computed by sequential one-dimensional row and column operations on a data array. As indicated by Table 5.2-1, a considerable savings in computation is possible for such transforms: computation by Eq 5.2-2 in the general case requires operations; computation by Eq. 5.2-10, when it applies, requires only operations. Furthermore, F may be stored in a serial memory and fetched line by line. With this technique, however, it is necessary to transpose the result of the col- umn transforms in order to perform the row transforms. References 2 and 3 describe algorithms for line storage matrix transposition. T diag T C1 T C2 … T CN ,,,[]= T Cj T diag T C T C … T C ,,,[]T C I N ⊗== T mn diag T R1 mn,()T R2 mn,()…T RN mn,(),,,[]= T Rj T mn diag T R mn,()T R mn,()…T R mn,(),,,[]= TI N T R ⊗= TT C I N ⊗ I N T R ⊗+= M 2 N 2 MN 2 M 2 N+ IMAGE STATISTICAL CHARACTERIZATION 127 TABLE 5.2-1. Computational Requirements for Linear Transform Operator 5.3. IMAGE STATISTICAL CHARACTERIZATION The statistical descriptors of continuous images presented in Chapter 1 can be applied directly to characterize discrete images. In this section, expressions are developed for the statistical moments of discrete image arrays. Joint probability density models for discrete image fields are described in the following section. Ref- erence 4 provides background information for this subject. The moments of a discrete image process may be expressed conveniently in vector-space form. The mean value of the discrete image function is a matrix of the form (5.3-1) If the image array is written as a column-scanned vector, the mean of the image vec- tor is (5.3-2) The correlation function of the image array is given by (5.3-3) where the represent points of the image array. Similarly, the covariance function of the image array is (5.3-4) Case Operations (Multiply and Add) General N 4 Column processing N 3 Row processing N 3 Row and column processing 2N 3 – N 2 Separable row and column processing matrix form 2N 3 E F{} EFn 1 n 2 ,(){}[]= η ηη η f E f{} N n E F{}v n n 1 = N 2 ∑ == Rn 1 n 2 n 3 n 4 ,;,()EFn 1 n 2 ,()F ∗ n 3 n 4 ,(){}= n i Kn 1 n 2 n 3 n 4 ,;,()EFn 1 n 2 ,()EFn 1 n 2 ,(){}–[]F ∗ n 3 n 4 ,()EF ∗ n 3 n 4 ,(){}–[]{}= 128 DISCRETE IMAGE MATHEMATICAL CHARACTERIZATION Finally, the variance function of the image array is obtained directly from the cova- riance function as (5.3-5) If the image array is represented in vector form, the correlation matrix of f can be written in terms of the correlation of elements of F as (5.3-6a) or (5.3-6b) The term (5.3-7) is the correlation matrix of the mth and nth columns of F. Hence it is possi- ble to express in partitioned form as (5.3-8) The covariance matrix of f can be found from its correlation matrix and mean vector by the relation (5.3-9) A variance matrix of the array is defined as a matrix whose elements represent the variances of the corresponding elements of the array. The elements of this matrix may be extracted directly from the covariance matrix partitions of . That is, σ 2 n 1 n 2 ,()Kn 1 n 2 n 1 n 2 ,;,()= R f E ff ∗ T {}E N m Fv m m 1 = N 2 ∑    v n T F ∗ T N n T n 1 = N 2 ∑       == R f N m E Fv m v n T F ∗ T    N n T n 1 = N 2 ∑ m 1 = N 2 ∑ = E Fv m v n T F ∗ T    R mn = N 1 N 1 × R f R f R 11 R 12 … R 1N 2 R 21 R 22 … R 2N 2 R N 2 1 R N 2 2 … R N 2 N 2 = … … … K f R f η ηη η f η ηη η f ∗ T –= V F Fn 1 n 2 ,() K f IMAGE STATISTICAL CHARACTERIZATION 129 (5.3-10) If the image matrix F is wide-sense stationary, the correlation function can be expressed as (5.3-11) where and . Correspondingly, the covariance matrix parti- tions of Eq. 5.3-9 are related by (5.3-12a) (5.3-12b) where . Hence, for a wide-sense-stationary image array (5.3-13) The matrix of Eq. 5.3-13 is of block Toeplitz form (5). Finally, if the covariance between elements is separable into the product of row and column covariance func- tions, then the covariance matrix of the image vector can be expressed as the direct product of row and column covariance matrices. Under this condition (5.3-14) where is a covariance matrix of each column of F and is a covariance matrix of the rows of F. V F n 1 n 2 ,()K n 2 n 2 , n 1 n 1 ,()= Rn 1 n 2 n 3 n 4 ,;,()Rn 1 n 3 – n 2 n 4 –,()Rjk,()== j n 1 n 3 –= kn 2 n 4 –= K mn K k = mn≥ K ∗ mn K ∗ k = mn< kmn– 1+= K f K 1 K 2 … K N 2 K ∗ 2 K 1 … K N 2 1 – K ∗ N 2 K ∗ N 2 1 – … K 1 = … … … K f K C K R ⊗ K R 11,()K C K R 12,()K C … K R 1 N 2 ,()K C K R 21,()K C K R 22,()K C … K R 2 N 2 ,()K C K R N 2 1,()K C K R N 2 2,()K C … K R N 2 N 2 ,()K C == … … … K C N 1 N 1 × K R N 2 N 2 × 130 DISCRETE IMAGE MATHEMATICAL CHARACTERIZATION As a special case, consider the situation in which adjacent pixels along an image row have a correlation of and a self-correlation of unity. Then the covariance matrix reduces to (5.3-15) FIGURE 5.3-1. Covariance measurements of the smpte_girl_luminance mono- chrome image. 0.0 ρ R 1.0≤≤() K R σ R 2 1 ρ R …ρ R N 2 1 – ρ R 1 …ρ R N 2 2 – ρ R N 2 1 – ρ R N 2 2 – … 1 = … … …