Linear Minimum Mean-Square-Error Transceiver Design for Amplify-and-Forward Multiple Antenna Relaying Systems

Linear Minimum Mean-Square-Error Transceiver Design for Amplify-and-Forward Multiple Antenna Relaying Systems

Linear Minimum Mean-Square-Error

Transceiver Design for Amplify-and-Forward

Multiple Antenna Relaying Systems

submitted byChengwen Xingfor the degree of Doctor of Philosophy

at The University of Hong Kong

July 7, 2010

Multiple-antenna communication system is an important research topic in thepast decades It increases the data rate or diversity in reception, without occupy-ing additional frequency or time resource On the other hand, amplify-and-forward(AF) relaying attracts a lot of attention lately, as it is suitable in cases where thesource cannot directly communicate with the destination, but is possible via a relay

in the middle The AF relay simply amplifies the received signal without decoding,thus its operation is favorable in implementation The combination of multiple-input multiple-output (MIMO) communication and AF relaying technique is cur-rently under consideration for several future wireless communication standards.With the source, relay and destination all equipped with multiple antennas, anatural question is how to allocate the limited power resource to make the commu-nication as efficient as possible This problem is addressed by linear transceiverdesign in this thesis Transceiver designs for point-to-point MIMO or multi-userMIMO systems have been widely addressed previously However, for AF MIMO

In this thesis, we start with a fundamental three nodes source-relay-destinationMIMO system The forwarding matrix at relay and equalizer at destination arejointly designed, under the realistic scenario that channel estimates in both hopcontains Gaussian error Two robust design algorithms are proposed to minimizethe mean-square-error (MSE) of the output signal at the destination The first one

is an iterative algorithm with its convergence proved analytically The other is anapproximated closed-form solution with much lower complexity than the iterativealgorithm

Next, we consider the AF MIMO orthogonal frequency division multiplexing(OFDM) system over frequency selective fading channels Again, the forwardingmatrix at relay and equalizer at destination are jointly designed by minimizing thetotal MSE of the output signal at the destination, under channel estimation errors.However, since OFDM is a multicarrier modulation, transceiver design in such sys-tem involves power allocation in both spatial and frequency domains, and thus ismore complicated than the first system In the proposed solution, the second-ordermoments of channel estimation errors in the two hops are first deduced in the fre-quency domain Then, the optimal designs for both correlated and uncorrelatedchannel estimation errors are investigated The relationship between the proposedsolutions with existing algorithms is also disclosed

responds to the case where one base station communicates with multiple terminalsvia one relay station In this system, the source precoder, relay forwarding matrixand destination equalizer are jointly designed by minimum MSE criterion Bothuplink and downlink cases are considered It is found that the uplink and downlinktransceiver designs share some common features and can be solved by a generaliterative algorithm On the other hand, another proposed algorithm for fully loaded

or overloaded uplink system is shown to include several existing results as specialcases

(Total words: 477)

Chengwen Xing

Transceiver Design for Amplify-and-Forward

Multiple Antenna Relaying Systems

by

Chengwen Xing

B.Eng., Xidian University, Xi’an, P R China

A thesis submitted in partial fulfillment ofthe requirements for the degree of

Copyright c° 2010 Chengwen Xing

I declare that this thesis represents my own work, except where due ment is made, and that it has not been previously included in a thesis, dissertation orreport submitted to this University or to any other institution for a degree diploma

acknowledg-or other qualifications

Signed

Chengwen Xing

First, I am most grateful to my supervisors, Dr Yik-Chung Wu and Dr RickyYu-Kwong Kwok I am very lucky to have two very nice supervisors in my Ph.D.study I really appreciate Dr Kwok for giving me the opportunity to study in theUniversity of Hong Kong I am deeply indebted to Dr Wu for his patient guidanceand encouragement which acquaints me with signal processing and optimizationtheory The door of his office is always open for me, and he selflessly shares with

me his invaluable experience in research He taught me how to read a paper, how tofind a direction, how to write a paper and even how to be a teacher I am indebted

to Dr Wu for all his help, support and kind consideration This thesis would not bepossible without the help from him

I would like to express my sincere gratitude to Dr Shaodan Ma, for her helpfulreviews, insightful comments and selfless help on my research I would also like tothank Prof Tung-Sang Ng for his encouragement and help in my study

Moreover, I would like to thank my friends I have met in the University of HongKong over the years: Dr Tyrone Tai-On Kwok, Dr Fanglei Sun, Dr Gan Zheng,

Dr Menglong Jiang, Dr Xiaoshan Liu, Dr Yiqing Zhou, Dr Hongzheng Wang,Mei Leng, Lanlan He, Dr Carson Ka-shun Hung, Dr Jianwu Chen, Dr Chiu-Wa

Ng, Xiao Li, Jun Zheng, Kun Cai, Xun Cai, Steve, Terry, Jing Xie, Jian Du, RuiMin and Bin Luo for their kindly help

I would like to thank the University of Hong Kong for providing financial port via postgraduate studentship and CRCG conference grants I must also ac-knowledge the Department of Electrical and Electronic Engineering for all its sup-port to postgraduate students, including the departmental conference grant.

sup-Finally, I am seriously indebted to my parents and my wife for their love Mywife, Ms Xingyuan Hao has been waiting for me for 5 years in Beijing during mypostgraduate study Thanks for her love

Table of Contents

Page

Declaration i

Acknowledgments iii

List of Figures viii

Abstract x

1 Introduction 1

1.1 Background 1

1.1.1 Cooperative Communication 1

1.1.2 Multiple-Input Multiple-Output Systems 2

1.1.3 AF MIMO Relay Systems 4

1.2 Research Motivation and Problems to be Tackled 4

1.3 Organization and Contributions of the Thesis 8

1.4 Commonly Used Notations 9

2 Robust Transceiver Design for AF MIMO Relay Systems 11

2.1 Introduction 11

2.2 System Model 12

2.3 Problem Formulation 15

2.4 The Proposed Iterative Algorithm 17

2.4.1 Updating G given F 18

2.4.2 Updating F given G 18

2.4.3 Summary and convergence analysis 20

2.5 The Proposed Closed-Form Solution 21

2.6 Extension to Weighted MSE Criterion 27

2.7 Simulation Results and Discussions 29

2.7.1 Simulation Setup 29

2.7.2 Convergence Performance of Iterative Algorithm 30

Page 2.7.3 Effect of Estimation Error σ2

e 32

2.7.4 Effect of Correlation Coefficients, α and β 33

2.7.5 BER Performance 34

2.8 Conclusions 35

2.9 Proof of q(γ i+1) is monotonically decreasing and upper bound on γ i+1 37

2.10 Proof of MSEU (F) ≥ MSE(F) 38

2.11 Derivation of optimal ˜F 39

3 Robust Transceiver Design for AF MIMO OFDM Relay Systems 41

3.1 Introduction 41

3.2 System Model 42

3.3 Channel Estimation Error Modeling 44

3.4 Transceiver Design Problem Formulation 47

3.5 Proposed Closed-Form Solution 50

3.5.1 Uncorrelated Channel Estimation Error 56

3.5.2 Correlated Channel Estimation Error 58

3.6 Simulation Results and Discussions 60

3.7 Conclusions 65

3.8 Proof of (3.7) 66

3.9 Proof of (3.17) 66

3.10 Proof of Property 1 68

3.11 Proof of Property 2 70

3.12 Proof of Property 3 71

4 LMMSE Transceiver Design for AF MIMO Relaying Cellular Net-works 74

4.1 Introduction 74

4.2 Downlink Transceiver Design 75

4.2.1 System model and problem formulation 75

4.2.2 Proposed iterative algorithm 78

4.2.3 Summary and Initialization 81

4.3 Uplink Transceiver Design 82

4.3.1 System model and analogy with downlink design 82

4.3.2 Uplink transceiver design for fully loaded or overloaded systems 85

4.3.3 Special cases 90

4.4 Simulation Results and Discussions 91

4.5 Conclusions 98

5 Conclusions and Future Research 99

5.1 Conclusions 99

5.2 Future Research Directions 100

List of References 102

List of Figures

1.1 Cooperative communication system 2

1.2 MIMO system 3

1.3 Single-user AF MIMO relay system 4

1.4 Multi-user AF MIMO relay system 5

2.1 Amplify-and-forward MIMO relay diagram 12

2.2 Convergence performance of the iterative algorithm with α = 0, β = 0.2 and σ2 e = 0.002 . 30

2.3 The MSEs for the closed-form solution and the iterative algorithm for different β and σ2 e , when α = 0 and σ2 e = 0.002 . 31

2.4 Convergence behaviors of the iterative algorithm with different initial-izations, when α = 0.6, β = 0.5 and σ2 e = 0.001 . 32

2.5 The MSEs for the closed-form solution, the iterative algorithm and the algorithm based on estimated channels only for different σ2 e, when α = 0.6 and β = 0.45 . 33

2.6 The MSEs for the two proposed solutions and the algorithm based on estimated channels only for different α, when β = 0.45 and σ2 e = 0.005. 34 2.7 The MSEs for the two proposed solutions and the algorithm based on estimated channels only for different β, when α = 0.45 and σ2 e = 0.005. 35

2.8 The BERs for the proposed closed-form solution, iterative algorithm and the algorithm based on estimated channels only for different σ2 e, when α = 0.45 and β = 0.45 . 36

Figure Page3.1 Amplify-and-forward MIMO OFDM relaying 423.2 MSE of received signal at the destination for different σ2

e when α = 0. 623.3 MSE of received signal at the destination for different σ2

e when α = 0.4 . 633.4 MSE of received data at the destination for HSA, SPA and proposed

robust algorithm when α = 0.4 . 633.5 MSE of received data at the destination for different α . 643.6 BER of received data at the destination for different σ2

e when α = 0.5. 654.1 Amplify-and-forward MIMO relay downlink and uplink cellular systems 754.2 The convergence behavior of the proposed Algorithm 1 when N B = 4,

N R = 4 and N M,k = 2 with 2 users 924.3 Total MSEs of detected data of the proposed Algorithm 1 and subopti-

mal algorithms, when N B = 4, N R = 4, N M,k = 2 and P r /σ2

v=20dB 934.4 Total MSEs of detected data of the proposed Algorithm 1 with andwithout precoder design 94

4.5 The convergence behavior of Algorithm 2 for uplink when N B = 4,

N R = 4 and N M,k = 2 954.6 Total MSEs of detected data of Algorithm 2 and suboptimal algo-

rithms, when N B = 4, N R = 4, N M,k = 2 and P s /σ2

n=20dB 964.7 Total MSEs of the detected data of the Algorithm 1, Algorithm 2 withrelaxation and the algorithm proposed in [30] 97

Multiple-antenna communication system is an important research topic in thepast decades It increases the data rate or diversity in reception, without occupy-ing additional frequency or time resource On the other hand, amplify-and-forward(AF) relaying attracts a lot of attention lately, as it is suitable in cases where thesource cannot directly communicate with the destination, but is possible via a relay

in the middle The AF relay simply amplifies the received signal without decoding,thus its operation is favorable in implementation The combination of multiple-input multiple-output (MIMO) communication and AF relaying technique is cur-rently under consideration for several future wireless communication standards.With the source, relay and destination all equipped with multiple antennas, anatural question is how to allocate the limited power resource to make the commu-nication as efficient as possible This problem is addressed by linear transceiverdesign in this thesis Transceiver designs for point-to-point MIMO or multi-userMIMO systems have been widely addressed previously However, for AF MIMO

relaying system, due to the relaying operation, transceiver design becomes morechallenging.

In this thesis, we start with a fundamental three nodes source-relay-destinationMIMO system The forwarding matrix at relay and equalizer at destination arejointly designed, under the realistic scenario that channel estimates in both hopcontains Gaussian error Two robust design algorithms are proposed to minimizethe mean-square-error (MSE) of the output signal at the destination The first one

is an iterative algorithm with its convergence proved analytically The other is anapproximated closed-form solution with much lower complexity than the iterativealgorithm

Next, we consider the AF MIMO orthogonal frequency division multiplexing(OFDM) system over frequency selective fading channels Again, the forwardingmatrix at relay and equalizer at destination are jointly designed by minimizing thetotal MSE of the output signal at the destination, under channel estimation errors.However, since OFDM is a multicarrier modulation, transceiver design in such sys-tem involves power allocation in both spatial and frequency domains, and thus ismore complicated than the first system In the proposed solution, the second-ordermoments of channel estimation errors in the two hops are first deduced in the fre-quency domain Then, the optimal designs for both correlated and uncorrelatedchannel estimation errors are investigated The relationship between the proposedsolutions with existing algorithms is also disclosed

Finally, we consider the AF MIMO relaying system with multiple users It responds to the case where one base station communicates with multiple terminalsvia one relay station In this system, the source precoder, relay forwarding matrixand destination equalizer are jointly designed by minimum MSE criterion Bothuplink and downlink cases are considered It is found that the uplink and downlinktransceiver designs share some common features and can be solved by a generaliterative algorithm On the other hand, another proposed algorithm for fully loaded

cor-or overloaded uplink system is shown to include several existing results as specialcases

Generally speaking, there are three kinds of cooperative protocols: forward (AF), decode-and-forward (DF) and compress-and-forward (CF) For AFstrategy, the relay only amplifies and forwards the received signal from the source

amplify-and-to the destination, without knowing what it has received While for DF strategy,the relay will decode the received data and then retransmit it to the destination

Figure 1.1 Cooperative communication system

In compress-and-forward strategy, the data before retransmission at relay will becompressed first to reduce the redundance

Among these three protocols, AF is a relatively simple one It does not need

to know the detailed transmission information at source, such as the modulationscheme, channel coding strategy, or source coding standard All operations at relayare only taken on analogue signal level, so AF strategy is also named as analoguerelay scheme Because of its simplicity and low implementation complexity, AFstrategy has attracted a lot of researchers’ attention [5–14] Meanwhile, AF coop-erative communication has small time delay and high security, thus also welcomed

by wireless industries, and taken as a candidate of the relay strategies in futurecooperative communication protocols [3, 4]

1.1.2 Multiple-Input Multiple-Output Systems

It is well-known that in fully scattered environment, multi-antenna systems vide substantial spatial diversity and multiplexing gains [15–17] With multipleantennas at both transmitter and receiver, multiple data streams can be simultane-ously transmitted without any increase in frequency or time resources as shown

pro-in Fig 1.2 Multiple-pro-input multiple-output (MIMO) system is a great success pro-inwireless communication theory research in the past decades [15–22] It is also one

of the most important technologies for the third generation mobile communicationsystems and beyond

Source Destination

Figure 1.2 MIMO system

To exploit the benefits of MIMO systems, transceiver design for MIMO systems

is of great meaning and has been extensively studied [15–17, 19, 21, 22] Propertransceiver design can greatly improve the performance of a MIMO system andcan even reduce its sensitivity to channel estimation errors From different designpurposes or to tackle different problems, the criteria of transceiver design can bevarious In general, there are two criteria for transceiver design: capacity maxi-mization and data mean-square-error (MSE) minimization The first one mainlyfocuses on maximizing the throughput between source and destination Usually, itaims at the problem how much information can be transmitted over wireless chan-nel under a transmit power constraint On the other hand, the design based on thesecond criterion tries to solve the problem how accurate a transmit signal can berecovered from the received signal Their relationship has been revealed in a land-mark paper [23]: capacity maximization transceiver design can be interpreted as aminimum weighted MSE transceiver design

From signal processing perspective, in order to minimize the data estimation ror from the received signal, MSE is a very important metric for transceiver design[23–26] Furthermore, based on implementation consideration, linear minimummean-square-error (LMMSE) transceiver is more preferable compared to its non-linear counterparts which may have prohibitive complexity Therefore, LMMSE

er-Relay Source Destination

Figure 1.3 Single-user AF MIMO relay system

transceiver design for conventional MIMO systems has been extensively studied invarious scenarios in the past decade [23–26]

1.1.3 AF MIMO Relay Systems

The benefits of multiple-antenna systems can be directly introduced into AFcooperative communications via deployment of multiple antennas at the source, re-lay and destination, and such AF MIMO relay systems receive a lot of attentionlately However, due to the AF operation at relay, the transceiver design is verychallenging Furthermore, depending on the specific modulation and number ofusers, the transceiver design is of different natures and must be tackled in differentways In this thesis, we address the problem of transceiver design for single-user

AF MIMO relay systems, where there is only one source, one relay and one tion [13, 14, 27–32] and multiuser AF MIMO relay systems, where multiple mobileterminals communicate with base station via one relay station [33, 34]

destina-1.2 Research Motivation and Problems to be Tackled

1.2.0.1 Single-user AF MIMO relay systems

In a classical single-user AF MIMO relay system as shown in Fig 1.3, there

is one source, one relay and one destination This case corresponds to a practicalscenario when one mobile terminal wants to communicate with base station via the

Relay station Base station Mobile terminals

Figure 1.4 Multi-user AF MIMO relay system

help of a relay station It is the basis of the more complicated AF MIMO relaysystems

Capacity bounds under different channel state information (CSI) assumptionsfor single-user AF MIMO relay systems have been derived in [27], [28] Transceiverdesign maximizing the capacity of single-user AF MIMO relay systems based onperfect CSI assumption have been studied in [13, 14, 29]

In terms of transceiver design minimizing MSE, for dual-hop AF MIMO relaysystems with single relay, the optimal closed-form solution for joint optimal for-warding and equalizer matrices assuming perfect CSI has been proposed in [30] Ithas also been shown in [30] that the joint design has a better performance than thevarious separate design schemes Furthermore, the joint source precoder, relay for-warding matrix and destination equalizer design for dual hop MIMO-OFDM relaysystems under perfect CSI is proposed in [31] However, there is no closed-formsolution and an iterative algorithm is proposed

Notice that the existing algorithms on LMMSE transceiver design for user AF MIMO relay systems require the CSI to be perfectly known Unfortu-nately, in practice, CSI is generally obtained through estimation and perfect estima-tion is very difficult to achieve Due to limited length of training sequences and/or

single-time-varying nature of wireless channels, channel estimation errors inevitably ist, causing substantial system performance degradation Robust transceiver design,which could mitigate such performance degradation by taking the channel estima-tion errors into account, is therefore of great importance and highly desirable forpractical applications.

ex-When channel uncertainties are considered, both min-max and stochastic cluding both probability-based and Bayesian) approaches can be employed Ifquality-of-service (QoS) requirement is considered (e.g., outage probability level[35, 36]), min-max or probability-based approach is preferred On the other hand,

(in-if the goal is to minimize an average objective function over channel uncertainties,e.g., the total MSE of multiple data streams, Bayesian approach is more suitable

In this thesis, Bayesian robust LMMSE transceiver design for single user AFMIMO relay systems (including both single carrier and multi-carrier systems) isinvestigated Several existing algorithms can be considered as special cases of ourproposed solutions

1.2.0.2 Multi-user AF MIMO relay systems

One of the most important application scenarios of cooperative communications

is cellular network Due to shadowing or deep fading of wireless channels, the basestation may not be able to sufficiently cover all mobile terminals in a cell, especiallythose on the edge Deployment of relay stations is an effective and economic way

to improve the communication quality in cellular networks, as shown in Fig 1.4.With multiple antennas at mobile terminals, relay station and base station, anatural question is how to allocate limited power resource in the spatial domain

In general, power allocation is equivalent to beamforming matrices design at base

station, relay and mobile terminals, and the objective can also be maximizing pacity [37] or minimizing the MSE of the recovered data [38] The MSE criterion is

ca-a widely chosen one since it ca-aims ca-at the recovered dca-atca-a to be ca-as ca-accurca-ate ca-as possible,and is extensively used in transceiver design in classical point-to-point [16, 17, 23]

or multiuser MIMO systems [39–46] The MSE minimization is also related to pacity maximization in the multiuser scenario [44] if a suitable weighting is applied

ca-to different data streams

In a cellular network, the base station and relay station are usually allowed to

be equipped with multiple antennas For each mobile terminal, if it is equippedwith single antenna, such relay cellular networks has been investigated from vari-ous point-of-views For example, transceiver design for capacity maximization hasbeen considered in [33], and quality-of-service based transceiver design has beeninvestigated in [34] However, in the next generation multi-media wireless com-munications, it is likely that the size of a mobile terminal allows multiple antennas

to be deployed Unfortunately, extension from the previous works on single tenna mobile terminals to multi-antenna terminals is by no mean straightforward.The transceiver design becomes more challenging as it needs to allocate the poweramong different users and even among different data streams in the same user Themutual interference from different users should also be properly managed

an-In this thesis, LMMSE transceiver design for AF MIMO relaying cellular work is investigated, in which base station, relay station and mobile terminals areall equipped with multiple antennas Precoder at source, forwarding matrix at re-lay and equalizer at destination will be jointly designed Both uplink and downlinkscenarios are considered

net-1.3 Organization and Contributions of the Thesis

This thesis is organized as follows:

Chapter 2—Robust Transceiver Design for AF MIMO Relay Systems: In thischapter, the problem of robust linear relay precoder and destination equalizerdesign for a dual-hop AF MIMO relay system, with Gaussian random channeluncertainties in both hops is investigated By taking the channel uncertaintiesinto account, for AF MIMO relay systems two robust design algorithms areproposed to minimize the mean-square-error (MSE) of the output signal at thedestination One is an iterative algorithm with its convergence proved analyt-ically The other is an approximated closed-form solution with much lowercomplexity than the iterative algorithm Although the closed-form solutioninvolves a minor relaxation for the general case, when the column covariancematrix or the row coveriance of the channel estimation error at the secondhop is proportional to the identity matrix, no relaxation is needed and theproposed closed-form solution is the optimal solution

Chapter 3—Robust Transceiver Design for AF MIMO-OFDM Relay tems: In this chapter, joint design of linear relay precoder and destinationequalizer for dual-hop non-regenerative AF MIMO-OFDM systems underchannel estimation errors is investigated Second-order moments of chan-nel estimation errors in the two hops are first deduced Then based on theBayesian framework, joint design of linear robust precoder at the relay andequalizer at the destination is proposed to minimize the total mean-square-error (MSE) of the output signal at the destination The optimal designs for

Sys-both correlated and uncorrelated channel estimation errors are considered.The relationship with existing algorithms is also disclosed.

Chapter 4—LMMSE Transceiver Design for AF MIMO Relay Cellular works: In this chapter, linear transceiver design for AF relaying cellular net-works is considered, in which base station, relay station and mobile terminalsare all equipped with multiple antennas The design is based on minimumMSE criterion, and both uplink and downlink scenarios are considered It

Net-is found that the downlink and uplink transceiver design problems are in thesame form, and iterative algorithms with the same structure can be used tosolve the design problems For the specific cases of fully loaded or overloadeduplink systems, a novel algorithm is derived and its relationships with severalexisting transceiver design algorithms for conventional MIMO or multiusersystems are revealed Simulation results are presented to demonstrate theperformance advantage of the proposed design algorithms

Chapter 5—Conclusions and Future Research: We conclude our work in thischapter, and describe some challenges concerning future research directions

1.4 Commonly Used Notations

The following notations are used throughout this thesis Boldface lowercaseletters denote vectors, while boldface uppercase letters denote matrices The nota-tion ZHdenotes the Hermitian of the matrix Z, and Tr(Z) is the trace of the matrix

Z The symbol IM denotes an M × M identity matrix, while 0 M,N denotes an

M × N all zero matrix The notation Z 1/2 is the Hermitian square root of the itive semidefinite matrix Z, such that Z1/2Z1/2 = Z and Z1/2 is also a Hermitian

pos-matrix The operation diag{[A B]} denotes a block diagonal matrix with A and

B as block diagonal On the other hand, d{Z} denotes a vector consisting of the diagonal elements of Z The symbol E{.} represents the expectation operation.

The operation vec(Z) stacks the columns of the matrix Z into a single vector The

symbol ⊗ denotes the Kronecker product.

Chapter 2

Robust Transceiver Design for AF MIMO Relay Systems

2.1 Introduction

In this chapter, we consider an AF MIMO relay system with single relay and

address the problem of robust relay precoder and destination equalizer design under

imperfect CSI at both the relay and destination With the channel estimation errors

being modeled as Gaussian random variables, we employ the Bayesian approach

and robustness is incorporated into the optimization objective function in the form

of MSE averaged over the channel estimation errors Two robust design algorithms

are proposed The first one is an iterative algorithm with its convergence proved

analytically The other algorithm offers a closed-form solution with a lower

com-plexity than the iterative algorithm Although the closed-form solution involves a

minor relaxation for the general case, when the column covariance matrix of the

channel estimation error in the second hop is proportional to the identity matrix,

no relaxation is needed and the proposed closed-form solution is the optimal

so-lution Simulation results show that the two proposed robust transceivers perform

better than the transceiver without taking channel estimation errors into account

Relay Destination Source

Figure 2.1 Amplify-and-forward MIMO relay diagram

Furthermore, the proposed closed-form solution has a comparable performance tothe proposed iterative algorithm, but with a lower complexity

This section is organized as follows System model is presented in Section 2.2.The optimization problem for minimizing the total MSE is formulated in Sec-tion 2.3 In Section 2.4, an iterative algorithm is proposed to solve the optimizationproblem, while an approximated closed-form solution is given in Section 2.5 InSection 2.6, the iterative and closed-form solutions are further generalized to theweighted MSE criterion Finally, simulation results are given in Section 2.7 andconclusions are drawn in Section 2.8

2.2 System Model

In this chapter, a dual-hop amplify-and-forward (AF) cooperative

communica-tion system is considered In the considered system, there is one source with N S antennas, one relay with M R receive antennas and N R transmit antennas, and one

destination with M D antennas, as shown in Fig 2.1 At the first hop, the sourcetransmits data to the relay The received signal, x, at the relay is

where s is the N S × 1 data vector transmitted by the source with the covariance

matrix Rs = E{ssH} The matrix H sr is the MIMO channel matrix between the

source and the relay Symbol n1 is M R × 1 Gaussian noise vector with covariance

matrix Rn1 At the relay, the received signal x is multiplied by a precoder matrix

F, under a power constraint Tr(FRxFH) ≤ P r where Rx = E{xxH} and P r

is the maximum transmit power Then the resulting signal is transmitted to thedestination The received signal at the destination, y, can be written as

y = HrdFHsrs + HrdFn1+ n2, (2.2)where Hrdis the MIMO channel matrix between the relay and the destination, and

n2 is the additive Gaussian noise vector at the second hop with covariance matrix

Rn2 In order to guarantee the transmitted data s can be recovered at the destination,

it is assumed that M R , N R , and M D are greater than or equal to N S [10]

It is assumed that both the relay and destination have the estimated channel stateinformation (CSI) When channel estimation errors are considered, we have

unit variance The M R × M R matrix Σsr and N S × N S matrix ΨT

sr are the rowand column covariance matrices of ∆Hsr, respectively [16] It is easy to see thatvec(∆HT

sr ) ∼ CN (0 M R N S ×1 , Σ sr ⊗ ΨTsr ), where CN (m, C) denotes a complex

Gaussian random vector with mean m and covariance C Furthermore, the matrix

∆Hsr is said to have a matrix-variate complex Gaussian distribution, which can be

where the M D × M Dmatrix Σrd and N R × N Rmatrix ΨT

rdare the row and columncovariance matrices of ∆Hrd, respectively It is assumed that Hsr and Hrd areestimated independently, so the channel estimation errors, ∆Hsr and ∆Hrd, areindependent

Remark 1: In general, the expressions of Ψsr, Σsr, Ψrd and Σrd depend onspecific channel estimation algorithms If the channel estimation algorithm pro-posed in [49] is used, we have Ψsr = RT,sr, Σsr = σ2

e,srRR,sr, Ψrd = RT,rdand Σrd = σ2

e,rdRR,rd The matrices RT,sr and RR,sr are the transmit and ceive antennas correlation matrices at the source and the relay, respectively, and

in [49, 50], Ψsr, Σsr, Ψrd and Σrd are functions of the second-order statistics ofCSI, which can be considered to change very slowly and to be known a priori [5]

However, in the following, the proposed algorithms are developed without ing any specific form of Ψsr, Σsr, Ψrdand Σrd.

assum-2.3 Problem Formulation

At the destination, a linear equalizer G is adopted to detect the transmitted data

s (see Fig 2.1) The problem is how to design the linear precoder matrix F at therelay and the linear equalizer G at the destination to minimize the mean-square-error (MSE) of the received data at the destination:

MSE(F, G) = E{Tr¡(Gy − s)(Gy − s)H¢

where the expectation is taken with respect to s, ∆Hsr, ∆Hrd, n1 and n2

Since s, n1and n2are independent, the MSE expression (2.7) can be written as

For the inner expectation, due to the fact that the distribution of ∆Hsr is variate complex Gaussian with zero mean, the following equation holds [47]

Notice that the matrix Rxis the autocorrelation matrix of the receive signal x at therelay.

Subject to the transmit power constraint at the relay, the joint design of izer at the destination and precoder at the relay can be expressed as the followingoptimization problem

equal-min

F,G MSE(F, G) s.t Tr(FRxFH) ≤ P r (2.16)

If the channels are perfectly known without estimation errors, the problem (2.16)has been solved in [30] However, perfect channel estimation is difficult to achieve

in practice, due to limited training and time-varying nature of wireless channels.Channel estimation errors generally exist and the formulation (2.13) is a very com-plicated function of F and G, making the problem difficult to solve In the follow-ing, we propose two algorithms One is an iterative algorithm which solves (2.16)without any approximation The other is an approximated closed-form solution,which can be shown to provide a performance close to the iterative algorithm, butwith a much lower complexity

2.4 The Proposed Iterative Algorithm

In this section, we derive an iterative algorithm [40, 51] to solve for F and G

In the following, it is shown that given F, there is a closed-form solution for G,and vice versa Therefore, the proposed algorithm computes F and G iteratively,starting with an initial value

i+1 The KKT conditions can be shown to be

Fi+1= ( ¯HHrdGHi+1Gi+1H¯rd+ Tr(Gi+1ΣrdGHi+1)Ψrd + γ i+1I)−1H¯HrdGHi+1RsH¯HsrR−1x ,

iter-Obviously from (2.19a), in order to compute the optimal Fi+1, the Lagrangian

multiplier γ i+1should be calculated first However, there is no closed-form solution

of γ i+1simultaneously satisfying (2.19b) and (2.19c) [40] Below we propose a lowcomplexity method to solve (2.19b) and (2.19c) First, notice that in order to have

(2.19b) satisfied, either γ i+1 = 0 or Tr(Fi+1RxFH

i+1 ) = P r must hold If γ i+1 = 0

also makes (2.19c) satisfied, γ i+1 = 0 is a solution to (2.19b) and (2.19c) Sincegiven Gi+1, the optimization problem (2.16) is a convex quadratic programmingproblem of Fi+1, which has only one solution for Fi+1 , γ i+1= 0 is the only solution

to (2.19b) and (2.19c) in this case On other hand, if γ i+1= 0 does not make (2.19c)satisfied, we have to solve Tr(Fi+1RxFH

i+1 ) = P r It is proved in Appendix 2.9 thatwhen Gi+1is fixed, the function

i+1 ) ≤ P r, (2.19c) is satisfied automatically

in this case In summary, the proposed procedure for calculating γ i+1 is given asfollows:

2.4.3 Summary and convergence analysis

The proposed iterative algorithm proceeds between (2.18) and (2.19a), whichcan be summarized as Algorithm 2.1 This iterative algorithm can be shown toAlgorithm 2.1 The iterative algorithm for joint design of F and G

Initialize the algorithm with F0 where the N × N principle sub-matrix of F0

converge as follows It is obvious that when Fi is given, the objective function

in problem (2.16) is a convex quadratic function of Gi+1 The solution given

by (2.18) corresponds to the minimum MSE for the fixed Fi In other words,MSE(Fi , G i+1 ) ≤ MSE(F i , G i) On the other hand, when Gi+1 is obtained, theoptimization problem (2.16) is a convex quadratic programming problem of Fi+1,and the KKT conditions are the necessary and sufficient condition for the optimalsolution [52], so we have MSE(Fi+1 , G i+1 ) ≤ MSE(F i , G i+1) It follows thateach update on F or G will decrease the objective function and thus the iterativealgorithm converges

Although the iterative algorithm results in precoder and equalizer design thatgives satisfactory performances, it requires high complexity because of iterations.Furthermore, in practice, it is not known in advance how many iterations are needed

for the iterative algorithm to converge In the next section, we propose a form solution, which approximately solves (2.16) in the general case, but exactlywhen Ψrd ∝ I N R In the simulation, we find that the closed-form solution has acomparable performance to that of the iterative algorithm.

closed-Remark 2: As mentioned previously, when Gi+1 is given, the optimizationproblem (2.16) is a convex quadratic programming problem for Fi+1, which can

be reformulated as a semi-definite programming (SDP) problem and solved byinterior-point polynomial algorithms [54] But these algorithms have a much highercomplexity compared to the proposed algorithm based on KKT conditions (2.19a)-(2.19c)

2.5 The Proposed Closed-Form Solution

Since the constraint in the problem (2.16) does not involve the equalizer G, theoptimal G can be directly derived as a function of F, by differentiating the objective

function MSE(F, G) with respect to G ∗ and setting the result to zero This resultsin

ˆ

G = Rs( ¯HrdF ¯Hsr)H( ¯HrdFRxFHH¯Hrd+ K)−1 , (2.23)where K = Tr(FRxFHΨrd)Σrd+ Rn2 was previously defined in (2.15) Substi-tuting (2.23) into the MSE expression (2.13), the MSE can be written as