(Mw)⋅ −(y Mw*)=0 (6.55) ∴(Mw)T(y−Mw*)=0 . (6.56) In the second line, we have used the fact that a ã b = a T b . Since for any matrices A and B:
( )ABT =B AT T, (6.57) where T represents the transposition operator (write rows as columns, write columns as rows), the orthogonal solution reduces to:
w MT T(y−Mw*)=0 (6.58) wT(M yT −M MwT *)=0 . (6.59)
For this to hold, as long as w T is nonzero, the term in brackets ( M T y − M T Mw * ) must be zero, and so:
M MwT *=M yT . (6.60) Thus, the unique solution is:
w*=(M MT )−1M yT . (6.61) The quantity M + = ( M T M ) − 1 M T is also called the pseudoinverse. The following shows how to extend the previous linear - prediction example to implement this matrix - based solution.
y = x(3:N) ;
M = [x( 2:N − 1) x(1:N − 2)] ; w = inv (M ’ * M) * M ’ * y
w = 1.7222 −0.8297
This is essentially a different route to the same end point. The numerical result as described will be slightly different to the earlier numerical result using derivatives, due to the way the starting points for the sequence have been defi ned.
6.5 NOISE ESTIMATION AND OPTIMAL FILTERING
We now extend the above and apply the basic concepts to the problem of removing noise from a signal, where we have access to an estimate the noise. Such an approach is called a Wiener fi lter. 1 Consider the problem where we want to minimize the noise in a signal, and we are able to sample the noise component separately. One received channel will comprise the signal plus noise, the other will consist predominantly of
1 Named after Norbert Wiener.
c06.indd 183
c06.indd 183 4/13/2011 5:22:52 PM4/13/2011 5:22:52 PM
184 CHAPTER 6 TEMPORAL AND SPATIAL SIGNAL PROCESSING
noise only. Of course, the noise component is not precisely, sample - by - sample, identical to that received by the signal sensor. If it were, the problem would reduce to a simple sample - by - sample subtraction.
The situation is illustrated in Figure 6.9 . The noise source is correlated to some degree with the measured signal, and we represent this coupling via a linear transfer function. If we can estimate the linear transfer function, we can estimate the noise as received by the signal sensor, and thus subtract it out.
In the diagram, the output signal ˆ( )s n at instant n is
ˆ( )s n =y n( )−w xT ( ),n (6.62) where the weight vector w is
w=
⎛
⎝
⎜⎜
⎜⎜
⎞
⎠
⎟⎟
⎟⎟
−
w w wL 0 1
1
, (6.63)
and the sample vector x n is defi ned as the block of L samples starting at instant n
xn
x n x n x n L
= −
− +
⎛
⎝
⎜⎜
⎜⎜
⎞
⎠
⎟⎟
⎟⎟ ( )
( )
( )
1 . 1
(6.64)
As before, we square the output to get an indicative error power
ˆ ( )s n2 =( ( )y n −w xT ( ))n 2 (6.65) =y n2( )−2y n( )w xT ( ) (n + w xT ( )) .n 2 (6.66)
FIGURE 6.9 Wiener fi ltering for noise cancellation. The fi lter weights are updated for a block using both the noisy signal and the (approximately) noise - only signal. It is assumed that we do not have access to the clean signal, only the noise - contaminated signal.
Furthermore, the noise signal is only an estimate, and we use its statistical properties to determine the optimal fi lter.
y(n) = s(n) +v(n) Σ ˆs(n) = s(n) +v(n) − ˆv(n)
x(n) v(
Signal
Signal + Noise
Wiener filter
Recovered signal
Noise
Noise Coupling
n)
s(n) +
− ˆ v(n)
c06.indd 184
c06.indd 184 4/13/2011 5:22:52 PM4/13/2011 5:22:52 PM
6.5 NOISE ESTIMATION AND OPTIMAL FILTERING 185
Now:
ˆ( ) ( ) ˆ( ) ( ( ) ( )) ˆ( )
( ) ( ( ) ˆ( )).
s n y n v n s n v n v n s n v n v n
= −
= + −
= + −
(6.67)
Hence:
ˆ ( ) ( ( ) ( ) ˆ( ))
( ) ( )( ( ) ˆ( )) ( ( ) ˆ s n s n v n v n
s n s n v n v n v n v
2 2
2 2
= + −
= + − + − (( )) .n 2 (6.68)
Taking the average or mathematical expectation:
E s n{ ( )}ˆ2 =E s n{ ( )} 2 { ( ) ( )} 2 { ( ) ( )}2 + E s n v n − E s n v nˆ +E v n{ ( )−v nˆ( ))) } { ( )} {( ( ) ( )) }.
2
2 2
≈E s n +E v n −v nˆ (6.69)
For perfect cancellation, the noise estimate ˆ( )v n equals the noise v ( n ), and the last term on the right is zero. Since E { s 2 ( n )} is fi xed, minimizing the difference between noise and noise estimate corresponds to minimizing the output power E s n{ ( )}ˆ2 . So we must minimize the output power, and to do this, we take Equation 6.66 , and take the expectation over the left - and right - hand sides to give:
E s n{ ( )}2 =E y n{ ( )}2 −2E y n{ ( )w xT ( )}n +E{(w xT ( )) }n 2 (6.70) =σy2−2 { ( )E y nw xT ( )}n +E{w xT ( )nxT( ) }.nw (6.71) We have a number of vector - vector and scalar - vector products here, 2 so for simplic- ity, let us defi ne the vector and matrix products:
r=E y n x n{ ( ) ( )} (6.72) R=E{ ( )x nxT( )}. n (6.73) So then the cost function J that we need to minimize is:
J=E s n{ ( )}ˆ2 (6.74) =σ2y−2r wT +w Rw. T (6.75) Using the identities
∂
∂w( )w rT =r (6.76)
∂
∂ =
w(w RwT ) 2Rw, (6.77) the derivative becomes
∂
∂J = − +
w 2r 2Rw. (6.78)
Setting this to zero to fi nd the minimum - power solution:
w=R r−1 . (6.79)
2 Recall that scalars are italic font, column vectors are lower case bold typeface, and matrices bold uppercase.
c06.indd 185
c06.indd 185 4/13/2011 5:22:52 PM4/13/2011 5:22:52 PM
186 CHAPTER 6 TEMPORAL AND SPATIAL SIGNAL PROCESSING
Thus, we have a matrix solution to the estimation problem. The solution relies, again, on correlations formed as a vector and matrix. We can employ the reshape () function in MATLAB to aid in forming the matrices, as shown in Listing 6.2 . In this example, we have a noise - free sinusoid which is not visible, and a noise - corrupted sinusoid which is visible. The noise is also measured separately, although of course it is only an estimate of the noise. Note how the reshape () operator is used to form the sequence of x ( n ) and y ( n ) vectors, as matrices.
Listing 6.2 Testing the Wiener Filter Theory
N = 10,000; % number of samples in test x = randn (1, N); % noise in system
n = 0:N − 1;
yc = 2.0 * sin (2 * pi * n/(N − 1) * 5); % clean signal (not visible)
% noise coupling — unknown transfer function b = [1 0.8 −0.4 0.1];
v = fi lter (b, 1, x);
y = yc + v;
L = 4; % coeffi cient length
% Wiener optimal solution
% reshape signal vectors in order to calculate covariances ym = reshape (y, L, N/L); % the output (y) vector
xm = reshape (x, L, N/L); % the input (x) vector
xv = xm(1 ,:); % y vector at intervals corresponding % to the start of the xm ’ s above R = (xm * xm ’ )/(N/L); % E(X X ˆ T)
r = (ym * xv ’ )/(N/L); % E(y X)
% optimal weights wopt = inv (R) * r;
% noise estimate
vest = fi lter (wopt, 1, x);
% error = signal — noise estimate e = y − vest;
wopt =
1.0405 0.7406 −0.3747 0.0843
The result is that wopt forms an estimate of b. This is shown for a block of noise - corrupted data in Figure 6.10 . The correlations are formed for the block of input data samples.
The process of fi nding the optimal solution can be imagined in two dimensions as shown in Figure 6.11 as searching for the bottom of a bowl - shaped surface, where the two orthogonal axes form the weight vectors, and the cost function forms the third axis.
c06.indd 186
c06.indd 186 4/13/2011 5:22:52 PM4/13/2011 5:22:52 PM
6.5 NOISE ESTIMATION AND OPTIMAL FILTERING 187
FIGURE 6.10 Wiener fi ltering example. The clean signal is what we desire, but it is not available to us. The noisy signal is available, and after the optimal fi lter is applied, we have the signal estimate.
Clean signal
Noisy signal
Signal estimate
FIGURE 6.11 The performance surface in terms of the fi lter coeffi cients. For two coeffi cients, we can imagine the error surface as a bowl - shape with the cost function as the vertical axis. The goal is to fi nd the minimal cost, and hence the corresponding fi lter weights.
−2 −1 0 1 2
−2
−1 0 1 20
2 4 6 8 10 12 14
Weight w0
Performance surface for 2-tap filter (w0= 0.8, w1= −0.4)
Weight w1
Mean squared error
c06.indd 187
c06.indd 187 4/13/2011 5:22:52 PM4/13/2011 5:22:52 PM
188 CHAPTER 6 TEMPORAL AND SPATIAL SIGNAL PROCESSING
FIGURE 6.12 The tomography problem and one possible approach using the
backprojection algorithm. The so - called head phantom (Shepp and Logan 1974) consists of ellipses of varying densities, intended to represent a cross - section of human tissue. This gives a basis on which to compare algorithms. The reconstructed interior projection is shown on the right.
True image matrix f(x, y) Backprojection image matrix b(x, y)