Consider a zero-mean random signal X(t) whose band is limited to B Hz. As it is a band-limited signal, it can be represented by a sequence of samples collected with the frequency of 2B Hz. On the basis of this sequence of samples the original random signal
+
n(t) (N0/2)
Yk t = k/2B Filter
B
B −B
−B f f
Channel
X(t) Y(t)
Figure 1.24 Scheme of the system with the band-limited channel and additive noise
X(t) can be recovered with a probability of 1. Assume that we analyze the transmission of the signal X(t) over the channel limited to B Hz in the time period of T seconds.
The number of analyzed samples is then equal ton=2BT. During transmission through this channel the signal is disturbed by Additive White Gaussian Noise (AWGN) with the power density ofN0/2. Figure 1.24 shows a scheme of such a system. Because the signal on the output of the filter is Y (t)=X(t)+ν(t), where ν(t) is a result of filtration of the noise ν(t)by the lowpass filter of bandwidth B Hz, at the output of the sampler at timing instantst =k/2B we receive
Yk=Xk+Nk (1.102)
In Chapter 3 we will prove that the noise sampleNk is zero-mean and has varianceσ2= N0B. Subsequent samples are mutually uncorrelated, and as they are Gaussian they are also statistically independent. Let statistical independence also be a feature of samplesXk of the input signalX(t). Thus, transmission of signalX(t)through the channel limited to BHz during the period ofT seconds can be treated asn=2BT independent transmissions of samplesXk through the discrete memoryless channel described by expression (1.102), often called the discrete time Gaussian memoryless channel. Let us note that all the variables appearing in formula (1.102) are continuous random variables:Nkis a Gaussian variable and Xk has the probability distribution pX(x). A natural assumption is that the mean power of the input signal is finite, i.e.
E[X2k]=P , k=1,2, . . . , n (1.103) Let us define the capacity of a discrete memoryless Gaussian channel as
Cs = max
pX(x)
*I (Xk;Yk): E[X2k]=P+
(1.104) where I (Xk;Yk)=h(Yk)−h(Yk|Xk). In order to derive the capacityCs let us first cal- culateh(Yk|Xk):
h(Yk|Xk)=
$∞
−∞
$∞
−∞
pX,Y(x, y)log 1
pY(y|x)dxdy
=
$∞
−∞
pX(x)
$∞
−∞
pY(y|x)log 1 pY(y|x)dy
dx (1.105)
As we remember, the noise sample Nk has a Gaussian distribution with zero-mean and varianceσ2. Then the probability density function of the channel output sampleYkcondi- tioned on the occurrence of a specific value of the input sample Xk is Gaussian with the same variance, but with the mean equal to the value of the sampleXk. The conditional probability density functionpY(y|x)is therefore described by the formula
pY(y|x)= 1
√2π σ exp
"
−(y−x)2 2σ2
#
(1.106) Using the latter expression in formula (1.105), we obtain
h(Yk|Xk)=
$∞
−∞
pX(x)
$∞
−∞
pY(y|x)
log(√
2π σ )+(y−x)2 2σ2 loge
dy
dx
=
$∞
−∞
pX(x)
log(
2π σ2ã1+loge 2σ2σ2
dx= 1
2log 2π eσ2
(1.107)
As we see, the conditional differential entropy depends exclusively on the noise variance and it does not depend on the distribution of the samplesXk of the input signal. Thus, in order to maximize the value of mutual informationI (Xk;Yk)it is necessary to maximize the entropy of samples Yk of the channel output signal. As we have already proven, the differential entropy of a continuous random variable achieves its maximum if the variable is Gaussian. The sample Yk, which is the sum of two random variablesXk and Nk, has a probability density function that is a convolution of the Gaussian distribution of the noise sample Nk and the probability distribution of the input signal sampleXk. Fortunately, if the probability density function of the sample Xk is Gaussian, then the probability density function of the sample Yk is also Gaussian, because a convolution of two Gaussian curves is also Gaussian. Concluding, the differential entropyh(Yk)achieves its maximum if the random signalX(t) and its samples are Gaussian. Since the samples Xk andNk are statistically independent, the mean power of their sum is equal to the sum of mean powers of both components. So on the basis of formula (1.97) for the maximum differential entropy we have
h(Yk)= 1 2log
2π e(P +σ2)
(1.108) As a result, the capacity of a time-discrete Gaussian memoryless channel is
Cs = 1 2log
2π e(P +σ2)
−1 2log
2π eσ2
= 1 2log
1+ P
σ2
(1.109) So far we have calculated the capacity for a single sample only. Since in the time period T there are n=2BT samples, the capacity of the channel band-limited to B Hz and
disturbed by additive Gaussian noise with the power densityN0/2 can be calculated from the formula
C= n
TCs = 2BT T ã1
2log
1+ P σ2
=Blog
1+ P N0B
[bit/s] (1.110)
The above calculations allow us to formulate the following theorem.
Theorem 1.13.1 The capacity of the channel band-limited toBHz, in which the signal is disturbed by the additive Gaussian noise of the power density equal toN0/2, is described by the formula
C=Blog
1+ P N0B
[bit/s] (1.111)
whereP is the mean power of the transmitted signal.
The above theorem defines an essential limit for the data rate of errorless transmission in a Gaussian band-limited channel with the input signal of a limited power. Let us note that in order to approach the limit established by (1.111) as closely as possible, the transmitted signal should be Gaussian. At first glance this requirement seems to be difficult to fulfill, however probability distributions of some digital signals are well approximated by the Gaussian distribution. Figure 1.25a plots the channel capacity per Hz (measured in bit/s/Hz), versus signal-to-noise ratio (SNR, in dB). We see that the capacity per Hz
C/B [bit/s/Hz]
0 5 10 15 20 25 30
1 3 5 7 9
SNR [dB]
x 105 C x 105
0 2 4 6 8 10
B [Hz]
0 1 2 3 4 [bit/s]
(a) (b)
Figure 1.25 Channel capacity plot: (a) per spectrum unit versus the signal-to-noise ratio (SNR);
(b) for constant power of the input signal atP /N0=3×105versus bandwidth of the signal
increases almost linearly for high SNR on the decibel scale. At the assumption of a constant bandwidth, the capacity increases linearly along with the increase of transmitted signal power. Assume now that the mean power of the transmitted signal is constant but the channel bandwidthB changes. Figure 1.25b shows the channel capacity as a function of the channel bandwidth whenP /N0=3×105. As we see, the capacity increases along with the increase of the channel and transmitted signal bandwidth, however the rise has a shape similar to the curve of 1−e−αx type. It turns out that for the constant mean power of the transmitted signal and the increasing bandwidth the channel capacity tends to the asymptotic value equal to NP
0log2e. A channel type in which a very large band is occupied at the constant value of P /N0 is applied in the so-called spread spectrum systems. They will be presented in one of the later chapters. In such systems, if the signal bandwidth is very large, the SNR can be established even below 0 dB.
Example 1.13.1 Let us calculate the theoretical capacity of the acoustic telephone channel with the passband in the range300–3400Hz for the following values of SNR: 10, 15, 20, 25, 30, 35 dB. Assume that the amplitude channel characteristic is flat in its passband, which is in fact far from reality. The selected SNR levels are related to the following values of P /(N0B)on the linear scale: 10, 31.62, 100, 316.23, 1000, 3162.28. Using these values successively in formula (1.111) for the bandwidth ofB=3100Hz we receive the following approximate values of the channel capacity, given in kbit/s: 10.7, 15.6, 20.6, 25.8, 30.9, 36.0. Currently, data transmission methods applied on acoustic telephone channels allow a 28.8 kbit/s data rate to be achieved if highly sophisticated transmission and reception algorithms are used. Therefore, such a value is close to the limit at the SNR of30dB. Let us note that the telephone modems offered on the market, which conform to ITU-T V.90 Recommendation, use in fact the subscriber loop channel in a different manner than a typical acoustic modem, which uses only the bandwidth of 3100 Hz; therefore, they achieve higher data rates.