Hanoi University of Science and Technology School of Electronics and Telecommunications Twodimensional systems & Mathematical preliminaries 2D systems and mathematical preliminaries
Trang 1Hanoi University of Science and Technology School of Electronics and Telecommunications
Twodimensional systems
&
Mathematical preliminaries
2D systems and mathematical preliminaries
*
1. Notations and definitions
2. Linear systems and shift invariance
3. Fourier Transform
4. ZTransform or Laurent series
5. Matrix theory and results
6. Block matrices and Kronecker products
7. Random signals
8. Some results from estimation theory
9. Some results from information theory
Trang 2Hanoi University of Science and Technology School of Electronics and Telecommunications
1. Notations and definitions
Ø 1D and 2D functions
ü 1D: f (x ), g (n ), d (x ), d (n )
ü 2D: f ( y x , ), g ( n m , ), d ( y x, ), d ( n m, )
Ø Separable forms of 2D functions
ü Dirac:
ü Kronecker:
ü rect (x,y), sinc(x,y), comb(x,y)
) ( ) ( ) ,
( x y d x d y
d = ×
) ( ) ( ) ,
( m n d m d n
Ø Special functions
ü Dirac delta:
ü Shifting:
ü Scaling:
ü Kronecker delta:
ü Shifting:
ü Rectangle:
ü Signum:
ü Sinc:
ü Comb:
ü Triagle:
1 ) ( lim
;
0 ,
0 ) (
¹
+
-
®
e
e
d x x x dx
) ( ' ) ' ( ) '
( x x x dx f x
ò
+
-e
e
d
a
x
( d
d =
î
í
ì
=
¹
=
0
1
0
0 )
(
n
n x
d
) ( ) ( )
( m n m f n
å
¥
¥
-d
ï
ï
í
ì
>
£
=
2 /
1
0
2 /
1
1 )
(
x
x
x rect
ï
î
ï
í
ì
<
-
=
>
=
0
1
0
0
0
1 )
(
x
x
x
x sign
å
¥
¥
-
-
= ( ) )
( x x n
x
x
x c
p
p sin ) ( sin =
ï
ï
í
ì
>
£
-
=
1
0
1
1 )
(
x
x
x
x tri
2D systems and mathematical preliminaries
*
2. Linear systems and shift invariance
Ø 2D linear systems
[ ] ×
H y ( m , n ) = H [ x ( m , n ) ]
) ,
( n m
x
ü Linear superposition property:
ü Impulse response:
ü Impulse response is called PSF: Input and output are positive quantities
ü In general: Impulse response can take negative or complex values
ü Region of support (RoS) of impulse response
ü Finite impulse response (FIR) and infinite impulse response (IIR): When
RoS is finite or infinite
[ ( ' , ' ) ]
) ' , '
; ,
( m n m n m m n n
h = d H - -
[ a 1 x 1 ( m , n ) + a 2 x 2 ( m , n ) ] = a 1 H [ x 1 ( m , n ) ] + a 2 H [ x 2 ( m , n ) ] = a 1 y 1 ( m , n ) + a 2 y 2 ( m , n ) H
Trang 3Hanoi University of Science and Technology School of Electronics and Telecommunications
Ø Output of a linear system:
û
ù
ê
ë
é
-
-
H
=
H
= ( , ) åå ( ' , ' ) ( ' , ' ) )
, (
' '
n
n
m
m
n
m
x
n
m
x
n
m
y
m n
d
=
Þ
' ' ' '
) ' , '
; , ( ) ' , ' ( )
' , ' ( ) ' , ' ( )
,
(
m n
m n
n
m
n
m
h
n
m
x
n
n
m
m
n
m
x
n
m
Ø Spatially invariant or shiftinvariant system: H d [ ( m , n ) ] = h ( m , n ; 0 , 0 )
[ ( ' , ' ) ] ( ' , ' ; 0 , 0 ) )
' , '
; ,
( m n m n m m n n h m m n n
) ' , ' ( ) ' , '
; ,
( m n m n h m m n n
Þ
ü The shape of impulse response does not change as the impulse
response moves about the (m,n) plan
ü Discrete convolution: ( , ) ( ' , ' ) ( ' , ' ) ( , ) * ( , )
' '
n
m
x
n
m
h
n
m
x
n
n
m
m
h
n
m
y
m n
=
-
-
= åå
ü Continuous convolution:
' ' ) ' , ' ( ) ' , ' ( )
, (
* ) , ( ) ,
( x y h x y f x y h x x y y f x x dx dy
¥
¥
-
¥
¥
-
-
-
=
2D systems and mathematical preliminaries
*
3. Fourier transform
Ø FT of a 1D function: f(x)
[ ] [ ] ò
ò
¥ +
¥
-
-
-
+¥
¥
-
-
=
Á
=
Á
=
Á
=
Á
x
x
x
x
px
px
d
e
F
F
x
f
dx
e
x
f
x
f
F
x
j
x
j
2
1
1
2
) ( )
( )
( :
) ( )
( )
( :
Ø FT of a 2D function: f(x,y)
ò ò
¥ +
¥
-
¥ +
¥
-
+
-
-
+¥
¥
- +¥
¥
-
+
-
=
Á
=
Á
=
Á
=
Á
2
1 ) (
2
2
1
2
1
1
1
) (
2
2
1
2
1
2
1
) , ( )
, ( )
, ( :
) , ( )
, ( )
, ( :
x
x
x
x
x
x
x
x
x
x
p
x
x
p
d
d
e
F
F
y
x
f
dxdy
e
y
x
f
y
x
f
F
y
x
j
y
x j
Trang 4Hanoi University of Science and Technology School of Electronics and Telecommunications
Performing a change of variables:
Ø Properties of FT
ü Spatial frequencies
ü Uniqueness
ü Separability
ò ò
ò ò
+¥
¥
-
- +¥
¥
-
- +¥
¥
- +¥
¥
-
+
-
ú
û
ù
ê
ë
é
=
2
1 , ) ( , ) ( , )
ü Frequency response and eigenfunctions of shift invariant systems
) ,
( y x
F
ò ò
+¥
¥
- +¥
¥
-
+
-
-
= ( ' , ' ) ' ' )
, ( x y h x x y y e 2 ( 1 ' 2 ' ) dx d y
' ,
' y y y
x
x
) (
2 1 2
:
where F = e j p x x + x y
) (
2
2
1
2
1
) , ( ) ,
( x y e j x y
H
=
Þ
2D systems and mathematical preliminaries
*
ü Convolution theorem
ü Inner product preservation
) , (
F ) , ( ) , G(
) , (
* ) , ( ) , ( x y = h x y f x y Þ x 1 x 2 = H x 1 x 2 × x 1 x 2
g
ü Correlation between 2 real functions
ò ò
+¥
¥
- +¥
¥
-
+ +
=
·
= ( , ) ( , ) ( ' , ' ) ( ' , ' ) ' ' )
,
( x y h x y f x y h x y f x x y y dx dy
c
Performing a change of variables:
) , F(
) , ( ) , (
C ) , ( ) , ( ) , ( x y = h - x - y · f x y Þ x 1 x 2 = H - x 1 - x 2 × x 1 x 2
c
ò ò
ò ò
+¥
¥
- +¥
¥
- +¥
¥
- +¥
¥
-
=
= f ( x , y ) h * ( x , y ) dxdy F ( x1 , x 2 ) H * ( x 1 , x 2 ) d x 1 d x 2
I
Setting h=f è Parseval energy conservation formula
ò ò
ò ò
+¥
¥
- +¥
¥
- +¥
¥
- +¥
¥
-
= 1 2 2 1 2
2
) , (
F )
,
f
Trang 5Hanoi University of Science and Technology School of Electronics and Telecommunications
ü Fourier transform pairs
comb(x,y) tri(x,y) rect(x,y)
1
) ,
( y x
) ,
( y x
d
yl
j
xk
j
e
e± 2 p ± 2 p
2
0
1 2
2 p x x j p y x
j e
) , ( x 1 m x k 2 m l
d
) ,
( x ± x 0 y ± y 0
d
) ( x 2 y 2
2 2
1 y
e - x p +
) , (
s inc x1 x 2
) , (
s 2 1 2
x
x inc
) , ( x1 x 2
comb
2D systems and mathematical preliminaries
*
Inner product
Spatial correlation
Multiplication
Convolution
Modulation
Shifting
Scaling
Separability
Conjungation
Linearity
Rotation
Property of 2D FT Function Fourier transform
) , F( 1 2
) (
2 0 1 0 2
x
x
x
x
j
) ( ) ( 2
1 x f y
f
) ,
( x y
) , (
F ) , (
H ) , (
G x 1 x 2 = x 1 x 2 × x 1 x 2
) , ( )
, ( 2 2
1
1 f x y a f x y
a + a 1F 1 ( x1 , x 2 ) + a 2 F 2 ( x 1 , x 2 )
) , (
*
y
x
f F *( - x - 1 , x 2 )
) (
F ) (
F 1 x 1 2 x 2
) ,
( y x
f
) ,
( ax by
) ,
( x x 0 y y 0
) , (
) (
2 1 2
y
x
f
) , ( ) , ( ) ,
( x y h x y f x y
) , ( ) , ( ) ,
( x y h x y f x y
g = × G ( x 1, x 2 ) = H ( x 1 , x 2 ) * F ( x 1 , x 2 )
) , ( ) , ( ) ,
( x y h x y f x y
c = · C ( x 1, x 2 ) = H ( - x 1 , - x 2 ) × F ( x 1 , x 2 )
ò ò
+¥
¥
- +¥
¥
-
+¥
¥
- +¥
¥
-
2
1
2
1
*
2
1 , ) H ( , ) (
F x x x x d x d x
) , (
F m x 11 m x 2
) , (
F x1 x 2
Trang 6Hanoi University of Science and Technology School of Electronics and Telecommunications
The evaluation of at and yields FT of
4. ZTransform or Laurent series
Ø Fourier transform of sequences (Fourier series): Selfreading
Ø Generalization of FT series: Ztransform
ü For 2D sequence x(m,n):
where z 1 , z 2 are complex variables
å å
+¥
-¥
= +¥
-¥
=
-
-
=
m n
n
m
z
z
n
m
x
z
z
ü Region of converge (RoC): this series converges uniformly in this region
ü Ztransform of a LSI system is called transfer function
) , (
) , ( ) , (
) , ( ) , ( ) , (
2
1
2
1
2
1
2
1
2
1
2
1
z
z
X
z
z
Y
z
z
H
z
z
X
z
z
H
z
z
Y
=
Þ
=
Ø Inverse Ztransform:
1
;
1 where ,
) , ( )
2 (
1 )
,
2
1
1
2
1
1
2
1
= ò ò X z z z - z - dz dz z z
j
n
m
p
) , ( z 1 z 2
1
w
j
e
2
w
j
e
2D systems and mathematical preliminaries
*
Ø Properties of 2D Ztransform
Multiplication
Convolution
Modulation
Shifting
Separability
Conjungation
Linearity
Rotation
) , ( 1 2
2
1
0
0
z
z
X
z
z ± m ± n
) ( ) ( 2
1 m x n
x
) ,
( m n
) , ( ) , ( z 1 z 2 F z 1 z 2
) , ( )
, ( 2 2
1
1 x m n a x m n
) , (
*
n
m
) (
F ) (
F 1 x 1 2 x 2
) ,
( y x
f
) ,
( m m 0 n n 0
) ,
( n m
x
b
a m n
) , ( ) ,
( m n x m n
) , ( ) ,
( m n y m n
x
) , ( z 1 - 1 z 2 - 1
X
) , (
F x 1 x 2
÷
ø
ö
ç
è
æ
b
z
a
z
,
ò ò ÷ ÷
ø
ö
ç
ç
è
æ
÷
ø
ö
ç
è
æ
1 2
'
2
2 '
1
1 '
2 '
1 '
2
2 '
1
1
2
) , ( ,
2
1
dz
z
dz
z
z
Y
z
z
z
z
X j
p
Trang 7Hanoi University of Science and Technology School of Electronics and Telecommunications
Ø Causality
ü Causal: Impulse response for and its transfer function
must have a onesided Laurent series
0 )
( = n
h n < 0
å
¥
=
-
=
0
) ( )
(
n
n
z
n
h
z
H
ü Anticausal: Impulse response for and its transfer function
must have a onesided Laurent series
0 )
( = n
ü Noncausal: Neither causal or anticausal
Ø Stability: Output remains uniformly bounded for any bounded input
¥
<
å
¥
=0
)
(
n
n
h
Ø Causal and stable system: poles of H(z) must lie inside the unit circle
Ø 2D case: å å < ¥ RoC of must include the unit circles
m n
n
m
2D systems and mathematical preliminaries
*
5. Matrix theory and results
Ø Vectors and matrices
ü Column vector of size N: U = u ( n ), n = 1 ¸ N
ü Row vector of size M: U = u ( m ), m = 1 ¸ M
ü Matrix A of size MxN containing M rows, N columns
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ë
é
=
) , ( )
2 , ( ),
1 , (
) ,
2 ( )
2 ,
2 ( ),
1 ,
2 (
) ,
1 ( )
2 ,
1 ( ),
1 ,
1
(
N
M
a
M
a
M
a
N
a
a
a
N
a
a
a
A
L
L
L
L
ü Index notation: A N ´ N = { a ( m , n ), 0 £ m , n £ N - 1 }
ü An image is usually visualized as a matrix è Exp. 2.2
Trang 8Hanoi University of Science and Technology School of Electronics and Telecommunications
Ø Row and column ordering
ü Rowordered vector (row stacking)
T
N
M
x
M
x
N
x
x
N
x
x
x
x = ( 1 , 1 ), ( 1 , 2 ), , ( 1 , ), ( 2 , 1 ), , ( 2 , ) L ( , 1 ), , ( , )
T
N
M
x
M
x
M
x
x
M
x
x
x
x = ( 1 , 1 ), ( 2 , 1 ), , ( , 1 ), ( 1 , 2 ), , ( , 2 ) L ( 1 , ), , ( , )
ü Columnordered vector (column stacking)
Ø Matrix theory definitions
{ a ( n m , ) }
A =
ü Matrix:
ü Transpose: A T = { a ( m n , ) } { * ( , ) }
*
n
m
a
A =
ü Complex conjungate:
{ ( m n ) }
I = d -
ü Conjungate transpose: ü Identity matrix:
ü Null matrix: O = { } 0
{ * ( , ) }
*
m
n
a
A T =
ü Matrix addition: A + B = { a ( m , n ) + b ( m , n ) } : A, B: Same dimension
ü Scalar multiplication: a = A { a a ( n m , ) }
ü Matrix multiplication: å
=
=
K
k
n
k
b
k
m
a
n
m
c
1
) , ( ) , ( )
,
2D systems and mathematical preliminaries
*
ü Vector inner product: X , Y = X * T Y = å x * ( n ) y ( n ) : Scalar quantity, if equal 0
è X and Y are orthogonal
ü Vector outer product: XY T = { x ( m ) y ( n ) } : X: Mx1, Y: Nx1, XY T : MxN
ü Symmetric: T
A
A =
ü Hermitian: A = A * T : Real symmetric matrix is Hermitan. Eigenvalues are real
ü Determinant: A
ü Rank of A: Number of independent rows or columns
ü Inverse matrix: A - 1 A = AA - 1 = I : Square matrix only
ü Singular: A 1 does not exist and A = 0
ü Eigen values : all roots of l k A - I l k = 0
ü Eigen vectors : all solutions of F k A F k = lk F k , F k ¹ 0
Trang 9Hanoi University of Science and Technology School of Electronics and Telecommunications
Ø Transpose and conjungate rules
[ ]
[ ]
[ ] [ ]
[ ] * * *
1
1
*
*
.
4
.
3
.
2
.
1
B
A
AB
A
A
A
B
AB
A
A
T
T
T
T
T
T
T
=
=
=
=
-
-
Ø Toeplitz and circulant matrices
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ë
é
=
-
-
+
-
-
+
-
-
0
1
2
1
1
2
2
1
0
1
1
1
0
, , ,
, ,
,
t
t
t
t
t
t
t
t
t
t
t
t
t
A
N
N
N
L
L
L
L
Ø Circulant matrix C
ú
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ê
ê
ë
é
=
-
-
-
-
0
1
2
1
2
2
1
0
1
1
2
1
0
, ,
,
, ,
,
,
c
c
c
c
c
c
c
c
c
c
c
c
c
C
N
N
N
N
L
L
L
L
C is also Toeplitz and c(m,n)=c((mn) modulo N) è Exp. 2.3
è Exp. 2.4
t(i,j) = t ij : Constant elements along the
main diagonal and subdiagonal
2D systems and mathematical preliminaries
*
where and are eigenvalues and eigenvectors of R
ü Other form ,which is the set of eigenvalue equations
Ø Orthogonal and unitary matrices
ü Orthogonal matrix:
ü Unitary matrix: A - 1 = A * T or AA * T = A * T A = I
è Exp. 2.5a
Ø Diagonal forms
ü If R is Hermitian matrix, there exists a unitary matrix Φ such that
where Λ is a diagonal matrix containing eigenvalues of R
L
=
F RΦ *T
L
= Φ
RΦ
N
k
k
k Φ , 1 , 2 , ,
RΦ k =l = L
{ } lk Φ k
è Exp. 2.5b
I
A
A
AA
A
Trang 10Hanoi University of Science and Technology School of Electronics and Telecommunications
Ø is block Toeplitz if is Toeplitz or
6. Block matrices and Kronecker products
Ø Block matrices of size : each element is a matrix itself
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ê
ë
é
=
À
n
m
m
m
n
n
A
A
A
A
A
A
A
A
A
,
2 ,
1 ,
,
2
2 ,
2
1 ,
2
,
1
2 ,
1
1 ,
1
,
, ,
L
L
L
L
where are matrices A , j
) (
, j A i j
Ø is block circulant if is circulant A i , j = A (( i - j ) mod ulo n ) ,m = n
èExp. 2.6 èExp. 2.7
Ø Kronecker products: A: M 1 xM 2 , B: N 1 xN 2 :
Ø Separable operations: selfreading
{ a m n B }
B
q
p ´
n
m ´
2D systems and mathematical preliminaries
*
7. Random signals
Ø Definitions: given a sequence of random variables u(n)
ü Mean:
ü Variance:
ü Covariance:
ü Crosscovariance:
ü Autocorrelation:
ü Crosscorrelation:
[ ( ) ]
) ( )
( n n E u n
u =m =
m
2
2
) ( ) ( )
( )
( n n E u n n
[ u ( n ), u ( n ' ) ] r ( n , n ' ) E { [ u ( n ) ( n ) ] [ u * ( n ' ) * ( n ' ) ] }
[ u ( n ), v ( n ' ) ] r ( n , n ' ) E { [ u ( n ) ( n ) ] [ v * ( n ' ) * ( n ' ) ] }
[ ( ) ( ) ] ( , ' ) ( ) ( ' ) )
' , ( ) ' , ( n n a n n E u n u * n r n n n * n
[ ( ) ( ) ] ( , ' ) ( ) ( ' ) )
' , ( n n E u n v * n r n n n * n
Ø For vector of size Nx1: u
[ ] { (n ) }
ü Cov [ ] u = E [ ( u - μ )( u * - μ * ) T ] = R u = R = { r ( n , n ' ) } is an NxN matrix
ü Cov [ ] u, v = E [ ( u - μ u )( v * - μ v * ) T ] = R uv = { r uv ( n , n ' ) } is an NxN matrix