Carrier recovery for high speed coherent optical communication systems based on digital signal processing

CARRIER PHASE RECOVERY FOR HIGHSPEED COHERENT OPTICAL COMMUNICATION SYSTEMS BASED ON DIGITAL SIGNAL PROCESSING XU ZHUORAN B.Eng., National University of Singapore, Singapore A THESIS SUB

CARRIER PHASE RECOVERY FOR HIGH

SPEED COHERENT OPTICAL

COMMUNICATION SYSTEMS BASED ON

DIGITAL SIGNAL PROCESSING

XU ZHUORAN

(B.Eng.), National University of Singapore, Singapore

A THESIS SUBMITTED FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE

2014

First of all, I would like to express my deepest gratitude and most sincere preciation to my supervisors Dr Changyuan Yu and Prof Pooi-Yuen Kam fortheir valuable guidance and kind support throughout my Ph.D study They en-lightened me by sharing their knowledge and experience in research This thesiswould not been possible without their help and encouragement

ap-I very appreciate for my thesis committee’s effort and time to review mythesis

Besides, I wish to thank my seniors Dr Zhang Shaoliang, Dr ZhangBanghong and Dr Shao Xuguang with whom I gain a lot of knowledge inresearch My thanks also go to my colleagues in NUS for providing friendlyenvironment and exchanging insightful ideas

Last but not least, I would like to thank my parents and relatives who arealways there to support me I am indebted for their love

1.1 Review of Coherent Optical Communication 2

1.2 Motivation 5

1.3 Contribution of the Thesis 9

1.4 Outline of the Thesis 11

2 Background 15 2.1 Advanced Modulation Formats 16

2.1.1 Signal Model 16

2.1.2 Generation Methods 17

2.2 Transmission Links 21

2.2.1 Linear Channel Impairments 21

2.2.2 Fiber Nonlinearity 24

2.3 Coherent Receiver 27

2.3.1 Coherent Detection 27

2.3.2 Signal-to-Noise Ratio 30

2.3.3 Laser Phase Noise 32

2.3.4 DSP Algorithms in Coherent Receivers 34

2.4 Conclusion 39

3 Trellis-Based Maximum Likelihood Sequence Detection 41 3.1 The Principle of Maximum Likelihood Sequence Detection 42

3.1.1 Signal Model 43

3.1.2 Conditional PDF of Carrier Phase 45

3.1.3 Decision Metric of Viterbi Algorithm 47

3.2 The Performance of MLSD in M PSK and M -QAM 51

3.2.1 Performance of MLSD in Linear Phase Noise Channel 51 3.2.2 Performance of MLSD in Long Haul Transmission System 64 3.3 Analysis of Phase Estimation Error 70

3.4 Conclusions 75

4 Adaptive MLSD Algorithm 77 4.1 The Principle of Adaptive MLSD 78

4.2 Simulation Performance of Adaptive MLSD 83

4.2.1 Performance of Adaptive MLSD in Linear Phase Noise Channel 83

4.2.2 Performance of Adaptive MLSD in Long Haul System 88

4.3 Experiment 92

4.3.1 Setup 92

4.3.2 Experimental results 95

4.4 Conclusion 100

5 Pilot Assissted MLSD Algorithm 103 5.1 The Principle of PA MLSD 104

5.2 Simulation Study of PA MLSD 107

5.3 Experiment 112

5.4 Conclusion 116

6 Conclusion and Future Work 119 6.1 Conclusions 119

6.2 Future work 121

6.2.1 Joint Estimation 121

6.2.2 Sequence Detection in OFDM 122

6.2.3 Multi-channel System 122

The exponentially growing demand of data traffic motivates the research of morespectrally efficient systems to better utilize the limited bandwidth of opticalfibers With the recent availability of high-speed analog-to-digital converters(ADCs), full information of electric field can be preserved, such as amplitude,phase and polarization Advanced modulation formats together with coherentdetection are shown to be the most promising solution to increase the data ratewithout increasing required bandwidth However, one of the challenges in co-herent optical systems is to recover the carrier phase, which is perturbed bylaser phase noise and nonlinear phase noise In this thesis, we will study carrierrecovery algorithms to compensate for phase noise impairments for coherentoptical systems

Firstly, a trellis-based maximum likelihood sequence detection algorithm isproposed and the implementable decision metric function is theoretically de-rived The conventional sequence detection is hard to be implemented since thecomputational complexity and memory requirement of the receiver increasesexponentially with the data sequence length Therefore, a Viterbi type scheme

is applied here to treat the sequence detection as a problem to search for thecorrect path through the trellis The survivor is chosen at each state among all

the incoming paths based on the decision metric function, which is the posteriorprobability given the received symbols.

Compared with conventional symbol-by-symbol (SBS) algorithms, the posed MLSD has superior performance In a channel with memory, the carrierphase of consecutive symbols are correlated, which gives an advantage for se-quence detection It is shown that the laser linewidth tolerance is tremendouslyincreased through Monte Carlo simulations However, the window length of thephase reference can influence the system performance If the window length

pro-is chosen to be long to average out the additive nopro-ise and accumulated ear phase noise, the estimation accuracy of the linear phase noise, which is fastvarying, will be compromised Optimal window length is theoretically anal-ysed However, the optimal performance can only be obtained when the systemparameters, such as phase noise variance, SNR and transmission distance, areavailable to the coherent receiver

nonlin-To solve this problem, the adaptive MLSD algorithm is proposed to matically deliver the optimal performance by self-adjusting the effective win-dow length A first-order adaptive filter is introduced to calculate the phasereference The filter gain coefficient can be viewed as a forgetting factor and iscontinuously updated based on received symbols It is shown that the adaptiveMLSD outperforms the non-adaptive version especially for MPSK signals ForQAM modulation format, although adaptive MLSD experiences constellationpenalty when the phase noise variance is very small, it approaches the optimumperformance when the phase noise becomes significant A back-to-back 100-Gbit/s polarization division multiplexing DQPSK experiment is demonstratedusing our proposed carrier recovery algorithm Significant improvements are

a pilot-assisted (PA) MLSD algorithm to eliminate the cycle slips by inserting

a short pilot between adjacent data blocks The phase reference is re-calculatedbased on the known pilot symbols after each data block Significant improve-ments are observed over symbol-by-symbol PA schemes Experiments of 10-GBaud/s 8QAM signals are demonstrated to verify our results

List of Tables

3.1 Complexity comparison among MLSD, DA ML PE and M thpower 513.2 Simulation parameters in long-haul transmission system 65

4.1 Complexity comparison among adaptive MLSD, MLSD and DA

ML PE 825.1 Complexity comparison among MLSD, PA DAML PE and PAMLSD 107

List of Figures

1.1 Photo of our testbed for 100G experiments 112.1 Block diagram of coherent optical communication system 162.2 The transfer curves of an MZM 192.3 The structure of I/Q modulator and QPSK constellation 19

2.4 Received signal constellation of QPSK signals transmitted over25×100km fiber at launched power of 0dBm 262.5 Structure of a single-polarization coherent receiver 28

2.6 Received constellation for 16QAM signal with SNR per bit of15dB and LLW=500kHz 33

2.7 Received constellation for 16QAM signal with SNR per bit of15dB and LLW=5MHz 34

2.8 Block diagram of the offline DSP blocks CDC: chromatic persion compensation; FOC: frequency offset compensation; CPE:carrier phase estimation 353.1 Uncoded QPSK trellis of data sequence of length K 443.2 BER performance comparison 40-Gbit/s DQPSK signals using

dis-DA ML PE and MLSD algorithms 52

3.3 QPSK BER performance of MLSD with different sequence lengthwith LPN, LLW=1MHz, 50Gb/s 55

3.4 16QAM BER performance of MLSD with different sequencelength with LPN, LLW=500kHz, 56Gb/s 56

3.5 BER performance comparison 40-Gbit/s D8PSK signals using

3.9 BER performance comparison of MLSD and DA ML PE for56-Gbit/s 16QAM signals with LLW=500kHz 62

3.10 BER performance comparison of MLSD and DA ML PE for56-Gbit/s 16QAM signals with LLW=1MHz 633.11 Block diagram of a long-haul optical transmission system 65

3.12 BER performance comparison between MLSD and DAML for50-Gbit/s DQPSK with transmission of 25×100km transmission 66

3.13 BER performance of MLSD and DAML for 50-Gbit/s DQPSKwith transmission of 25×100km transmission with different win-dow length 67

3.14 BER performance of MLSD and DAML for 50-Gbit/s DQPSKwith transmission of 25×100km transmission with different win-dow length 68

3.15 BER performance of MLSD and DAML for 50-Gbit/s DQPSKwith transmission of 25×100km transmission with different win-dow length 693.16 Phase estimation error variance vs memory length L 73

4.8 Q-factor of 50-Gbit/s DQPSK signal with different number of

fiber spans 90

4.9 BER performance comparison of 50-Gbit/s DQPSK signal with

transmission of 23×100km and LLW=10MHz 91

4.10 Experimental setup of the 100-Gbit/s PDM DQPSK system I:

in-phase data ; Q: quadrature data; PBS: polarization bean

split-ter; PBC: polarization beam combiner; VOA: variable optical

attenuator; Ix: in-phase X-polarization; Qx: quadrature X-polarization;Iy: in-phase Y-polarization; Qy: quadrature Y-polarization 92

4.11 Detail transmitter structure of the 100G experiment PPG: pulse

pattern generator; Mux: multiplexer; Clk: clock 944.12 Received signal constellation left: X-pol; right: Y-pol 954.13 Signal constellation after CMA filters left: X-pol; right: Y-pol 964.14 Recovered signal constellation after carrier phase estimation

left: X-pol; right: Y-pol 96

4.15 BER performance 100-Gb/s PDM DQPSK system with ECL

laser for X-Polarization 97

4.16 BER performance 100-Gb/s PDM DQPSK system with ECL laser for Y-Polarization 98

4.17 BER performance 100-Gb/s PDM DQPSK system with DFB laser for X-Polarization 99

4.18 BER performance 100-Gb/s PDM DQPSK system with DFB laser for Y-Polarization 99

5.1 Effect of the phase reference error on symbol decision for QPSK signal DB: decision boundary 104

5.2 Structure of the transmitted data for (a) MLSD; (b) PA DA ML; (c) PA MLSD.T: termination symbol 105

5.3 BER vs SNR performance comparison of 40-Gb/s 16QAM sig-nals for PA DAML and PA MLSD 108

5.4 BER performance comparison of 40-Gb/s 16QAM signals for PA DAML and PA MLSD with 6×100km transmission 109

5.5 Experiment setup of a 10-Gbaud/s B2B 8QAM system 110

5.6 Received S-8QAM constellation after re-sampling 111

5.7 Signal constellation after CMA filter 111

5.8 Recovered signal constellation of S-8QAM with Left: ECL laser; Right: DFB laser 113

5.9 Recovered signal constellation of R-8QAM with Left: ECL laser; Right: DFB laser 113

5.10 BER performance comparison between PA MLSD and M th-power with DE for S-8QAM signals with LLW=5MHz 114

LIST OF FIGURES

5.11 BER performance comparison between PA MLSD and M power with DE for R-8QAM signals with LLW=5MHz 115

th-List of Abbreviations

ADC Analog-to-Digital Converter

ASE Amplified Spontaneous Emission

ASIC Application-Specific Integrated CircuitASK Amplitude-Shift Keying

AWG Arbitrary Waveform Generator

AWGN AdditiveWhite Gaussian Noise

B2B Back-to-Back

BER Bit-Error Rate

BPS Blind Phase Search

BPSK Binary Phase Shift-Keying

CD Chromatic Dispersion

CDC Chromatic Dispersion Compensation

CMA Constant Modulus Algorithm

CMOS Complenentary Metal-Oxide-SemiconductorCPE Carrier Phase Estimation

DA Decision-Aided

DAC Digital-to-Analog Converters

DCF Dispersion-Compensating Fiber

ECL External Cavity Lasers

EDFA Erbium-Doped Fiber Amplifier

FEC Forward Error Control

FFT Fast-Fourier Transform

FIR Finite Impulse Response

GVD Group Velocity Dispersion

ICI Inter-Carrier Interference

IF Intermediate Frequency

IIR Infinite Impulse Response

IM/DD Intensity Modulation with Direct DetectionISI Intersymbol Interference

LDPC Low-Density Parity Check

LIST OF ABBREVIATIONS

MZM Mach-Zehnder Modulator

NLMS Normalized Least-Mean Square

NLPN Nonlinear Phase Noise

NRZ Non-Return-to-Zero

OFDM Orthogonal Frequency-Division Multiplexing

OOK On-Off Keying

OSNR Optical Signal-to-Noise Ratio

PAPR Peak-to-Average Power Ratio

PBS polarization beam splitter

PDM Polarization Division Multiplexing

Chapter 1

Introduction

Fiber-optic communication systems are lightwave systems that employ opticalfibers for information transmission The carrier frequencies for optical commu-nication systems are in the range of 100 THz which is the visible or near-infraredregion of the electromagnetic spectrum [1] Thus the available bandwidth forinformation transmission is much higher than in microwave systems To meetthe demand of fast increasing data traffic, fiber-optic communication systemshave been widely deployed since 1980 for metropolitan and trans-ocean com-munications Especially in the past two decades, the amount of data traffic onthe backbone networks has been growing exponentially at about 30 to 60% peryear [2] The rapid development of cloud computing also requires high speeddata communication within data-centers and high performance computers [3].The optical communication systems can be divided into two categories based

on their detection method One is referred to as intensity modulation with directdetection (IM/DD) The information is modulated on the intensity of an opticalcarrier After transmission through fiber link, the incident signal is converted

directly to electrical domain using photo detectors The capability of the directdetection system is highly limited since the phase information of the transmittedsignal is discarded.

The other scheme, referred to as coherent detection, modulates the tion onto both the amplitude and phase of the transmitted signal At the receiver,

informa-a locinforma-al oscillinforma-ator (LO) linforma-aser is informa-applied to beinforma-at with the incident light to informa-achievehomodyne or heterodyne detection Recent advances in high-speed analog-to-digital converters (ADCs) have revived the coherent detection systems Thefull information of the electromagnetic field can be preserved at the receiver bysampling the received signal into digital waveforms Thus the amplitude, phaseand polarization information of the signal can be modulated simultaneously toincrease the transmission capacity Advanced modulation formats such as M -ary phase-shift-keying (M -PSK) and M -ary quadrature-amplitude-modulation(M -QAM) can be employed to further increase system spectral efficiency (SE) Furthermore, various digital signal processing (DSP) techniques can be applied

to the digital samples to mitigate channel impairments of the fiber link fore, the bulky optical devices can be replaced by small and fast DSP circuits inthe coherent receiver

There-In this chapter, a brief review of the development of coherent optical munication is given, followed by the motivation and outline of this thesis

The coherent optical communication systems draw considerable attention andwere intensively investigated in 1980s The LO in the receiver dramatically im-

1.1 Review of Coherent Optical Communication

proves the receiver sensitivity and transmission distance tolerance of the opticalcommunication system [4] The optical phase lock loop (PLL) based meth-ods were employed to achieve carrier recovery However, this type of opticalmethod is too complex to implement in the receiver [5] Besides, the stability isanother drawback that limits the usage of optical PLL, since the product of thelaser linewidth of distributed feedback (DFB) lasers and loop delay is too large[6] It has been shown that delays greater than a few tens of nanoseconds wouldlead to loop instability even at a 10-Gbit/s system [7] As a result, heterodynedetection was introduced to ease the requirement of the feedback delay of op-tical PLLs An electrical PLL was used to track the phase of the intermediatefrequency (IF) signals at microwave frequency [8] However, the modulationformat was limited to simple ones such as binary phase shift-keying (BPSK) ,differential phase-shift-keying (DPSK) and amplitude-shift keying (ASK) Due to the mature optical amplifier and wavelength-division-multiplexing(WDM) technology, the attention was shifted back to intensity modulation/directdetection scheme in the 1990s [9, 10] The receiver sensitivity can be tremen-dously improved by employing erbium-doped fiber amplifiers (EDFAs) as pream-plifier [11] The transmission distance was extended up to thousands of kilome-ters by using cascaded EDFAs The capacity of optical communication systemswas increased to the order of Terabits per second with the aid of WDM tech-nologies in S-band(1460 to 1530 nm), C-band (1530 to 1565 nm) and L-band(1565 to 1625 nm) By the early 2000s, lasers had reached Gigahertz frequencystabilities, and optical filters had bandwidths allowing for 50-GHz WDM chan-nel spacings Therefore, the “optical and electronic bandwidths had met” [12].Communication engineering technology rather than physics became the main

factor that pack more information into the limited and most attractive bandwidth

of optical amplifiers BPSK and QPSK using direct detection with differentialdemodulation are investigated [11, 13] The self-homodyne receiver convertsthe phase difference between the current optical signal and its one-symbol de-layed version into optical intensity The scheme removes an LO laser that isalways present in a typical coherent receiver and consequently relaxes the laserlinewidth tolerance

Recent advances in high speed ADCs and DACs [14] have revived coherentoptical communication The state-of-the-art ADCs built in 40-nm complenen-tary metal-oxide-semiconductor (CMOS) technology reach the sample rate of

65 Gsample/s with an effective number of bits (ENOB) of about six [15] Withthe high speed ADCs employed at the coherent receiver, we can sample the in-cident photocurrents at the Nyquist rate or above so that the full information ofthe electric field, such as amplitude, phase and polarization, can be preserved.Such receiver is also referred to as digital coherent receiver [16] Therefore, ad-vanced modulation formats together with multiplexing technique can be applied

to increase the spectral efficiency tremendously Furthermore, DSP techniquescan be utilized in the receiver to perform all-electronic chromatic dispersion(CD) and polarization-mode dispersion (PMD) compensation, frequency off-set and phase noise mitigation, polarization de-multiplexing and so on [17–21].Therefore, the bulky optical components are replaced by compact and cheapDSP circuits in the receiver Single carrier 100-Gbit/s coherent optical systemsare commercially available in 2010 using PDM-QPSK at 28 GBaud, based oncustom-designed CMOS application-specific integrated circuit (ASIC) for vari-ous DSP functionality [22]

1.2 Motivation

To meet the requirement of high speed transmission up to Terabit/s and yond by 2015, optical superchannels that circumvent the electronic bottleneckvia optical parallelism are the method of choice [23] The term ”superchannel”

be-is first used in [24] refering to multiple single-carrier-modulated signals thatare seamlessly multiplexed under the coherent optical orthogonal frequency-division multiplexing (OFDM) conditions [25, 26] Later the concept of super-channel is generalized to the optical signals that are modulated and multiplexedtogether with high spectral efficiency at a common originating site, transmittedthrough common optical link and received at a common destination site TheNyquist-WDM with spectrally shaped single carrier modulated signals havealso been introduced [27] Currently, dual-carrier 400-Gbit/s digital coherenttransponders have been commercially available since 2013 [28] and long-haultransmissoin with Tb/s superchannel data rates has been experimentally demon-strated [29] Furthermore, since 2011, space-divisoin multiplexing (SDM) ex-periments using multi-core fiber have been demonstrated to beat single-modeaggregate per-fiber capacities [30,31] In addition to conventional square QAM,advanced modulation formats such as iterative polar modulation [32] and four-dimentional (4-D) modulatoin formats which jointly utilize the four orthogo-nal dimensions of two polarizations and two quadrants have also been demon-strated [33, 34]

As can be seen that coherent detection together with advanced modulation mats with multiplexing technique, such as PDM and SDM, can effectively im-

for-prove the spectral efficiency However, advanced modulation formats are lessrobust to distortions One of the challenges in coherent optical communication

is to recover carrier phase, which is corrupted by phase noise Unlike in wirelesscommunications where the frequency and phase changes are relatively similarand slow, the characteristics of frequency and phase offsets in a high-speed op-tical system are very different: the frequency change is relatively slow but therange can be very large up to 5GHz while the phase noise typically varies at

a much higher speed relative to the symbol rate [5] Optical PLL is difficult

to implement since the product of laser linewidth and loop delay is too large.Therefore, with recent advancement of high speed ADCs, DSP based methodshave been introduced and established themselves as the most promising solution

to address this challenge

The Viterbi & Viterbi M th power algorithm is widely-used to accuratelytrack the unknown phase noise [35] The received signal is first raised to the

M th-power to erase data modulation and the carrier phase is extracted through

a low-pass phase averaging filter with the assumption that the carrier phase doesnot change within a certain number of symbol intervals Apparently, this methodheavily depends on nonlinear operations such as rectangular to polar/polar torectangular transformations, M th power operation and phase unwrapping [20]

In addition, the M th power scheme is first proposed for M -PSK signals, thusextra modifications are required for non-constant-amplitude modulation for-mats [36, 37] To extend the M th-power scheme to M -ary QAM signals, wecan use a subgroup of symbols with phase modulation π/4 + nπ/2 (n=0, 1, 2,3) to estimate the carrier phase [38] Since only a fraction of received signalsare used to estimate the phase reference in a QAM system, its system perfor-

1.2 Motivation

mance is severely degraded, thus making the algorithm sensitive to laser phasenoise [39] A modified M th-power scheme in [40] suggests utilizing all the sym-bols by sub-partitioning those symbols belonging to the middle ring of 16QAMconstellation into two QPSK groups However, all these M th-power schemesrequire nonlinear operations, such as arctan(·) function, sub-grouping of sym-bols, and phase unwrapping

The blind phase search (BPS) algorithm is also introduced for coherent tical systems in [41] The carrier phase is scanned over a limited phase range([0, π/2] for a square-QAM) at fixed or variable phase increments, and the de-cisions made following each trial phase is approximated as the reference sig-nal for minimum mean square distance error calculation (MSDE) The optimalphase is the one that gives the minimum MSDE This algorithm can achievenearly optimum linewidth tolerance when the signal-to-noise ratio (SNR) ishigh However, the implementation complexity of this algorithm increases withthe modulation order Especially for high-order modulation format, the receivercomplexity can be very high

op-The decision-aided maximum likelihood phase estimation (DA ML PE) isproposed in [42] with only linear operations involved The performance of DA

ML PE is similar compared with M th-power scheme in linear phase noise nel With nonlinear phase noise as the main distortion, it outperforms M th-power scheme especially at the optimum launched power [43] However, theoptimal performance of this algorithm can only be obtained with channel statis-tics available to the coherent receiver Wiener filter [7] and Kalman filter [44]are also proposed for carrier phase estimation with the same prerequisite Anormalized least-mean square (NLMS) algorithm is also introduced to estimate

chan-the carrier phase with a fixed step size The optimum step size needs to be consumingly found out through either simulations or experiments, thus making

time-it not sutime-itable for online processing To solve this problem, an adaptive filter isintroduced [45] to estimate the phase noise without the need of channel statis-tics

Multistage phase recovery algorithms are also proposed such as [46] to duce the implementation complexity of the single-stage BPS algorithm, with ahybrid structure of BPS and ML recovery algorithm To further improve theaccuracy of phase estimation, more than one ML phase estimation stage can beadded to approach the optimal phase angle iteratively Training assisted or pi-lot assisted phase recovery algorithms are also introduced to mitigate the cycleslip problem [42, 47] Conventionally, differential encoding (DE) and decoding

re-is usually applied to prevent cycle slips and error propagation However, therewill be DE-induced performance penalty since one symbol decision error willlead to two symbol errors By using training symbols or pilots, we can removethe need of DE and thus improve system performance

With the usage of higher modulation formats, the phase noise tion capability of the coherent receiver becomes more critical Furthermore,with Nyquist wavelength division multiplexing (WDM) [27] and supper chan-nel techniques [29] employed to shrink the channel spacing to improve the SE,the transmitted signal will experience more distortions All these factors drivesthe motivation to develop carrier recovery algorithm which is more robust tosystem impairments Sequence detection is one promising candidate since itoutperforms symbol-by-symbol algorithms in channel with memory [48] How-ever, conventional sequence detection algorithms such as [49] are difficult to im-

compensa-1.3 Contribution of the Thesis

plement in the coherent receiver since the computational complexity increasesexponentially with sequence length

1.3 Contribution of the Thesis

In this thesis, a trellis-based maximum likelihood sequence detection algorithm(MLSD) is applied for carrier phase recovery and symbol detection in the co-herent receiver A Viterbi-type algorithm is applied to treat the sequence de-tection as a problem to searching for paths through the trellis [50] An im-plementable decision metric function is theoretically derived from the originalconditional probability function to choose the survivor at each state of the trel-lis Compared with conventional sequence detection method [49], the compu-tational complexity is fixed and effectively reduced The performance of theproposed MLSD algorithm is investigated in both linear phase noise dominantchannel (back-to-back system) and long-haul transmission system The BERperformance is superior compared with symbol-by-symbol algorithms such as

DA ML PE It is observed that MLSD significantly improves the BER mance especially when the received signals are seriously corrupted by the phasenoise, where the laser linewidth, i.e., linear phase noise variance, is very large.The laser linewidth tolerance at 1-dB SNR penalty compared with coherent de-tection is successfully increased by around four and two times for 50-Gbit/sDQPSK and 25-Gbaud/s D8PSK signals respectively For 16QAM signals, thelaser linewidth toleration is also successfully improved by 2 times from 250kHz

perfor-to 500kHz compared with DA ML PE

We also analytically study how the window size of the phase reference in

MLSD algorithm would influence the system performance It is shown thatthere is trade-off between averaging the AWGN noise, nonlinear phase noiseand tracing the fast varying linear phase noise The optimum window size istheoretically obtained However, we need the channel statistics such as the phasenoise variance and SNR.

To solve this problem, we also introduce the adaptive MLSD which can matically deliver the optimal performance without the pre knowledge of channelstatistics A first order filter is applied to estimate the carrier phase instead ofusing a sliding window The filter gain, which is obtained to minimize the esti-mation error function, can be regarded as a forgetting factor which determinesthe effective memory length of the phase reference The filter gain is recursivelyupdated based on the received symbol Both MLSD and its adaptive versionare modulation format transparent and only involve linear operations However,

auto-by using adaptive MLSD, the memory requirement of the coherent receiver isreduced from O(L · M ) to O(M ), where L is the window size for MLSD Simu-lations for back-to-back and transmission systems are conducted to compare theperformance for adaptive and non-adaptive MLSD algorithms It is observedthat the adaptive version achieves optimal performance as MLSD with optimumwindow size For QAM signals, the adaptive MLSD experiences constellationpenalty which leads to performance degradation especially in low BER region.Experiments of B2B 100-Gb/s PDM DQPSK signals as shown in Fig 1.1 arecarried out to verify our results

In addition, pilot-assisted MLSD is proposed to prevent cycle slips withoutthe need of DE Pilot symbols are inserted between consecutive data blocks toreset the phase reference before it is corrupted by decision errors 16 QAM

1.4 Outline of the Thesis

Figure 1.1: Photo of our testbed for 100G experiments

modulation format, which is one of the promising candidates for the future400G system, is simulated in linear phase noise channel and long haul trans-mission system Compared with the symbol-by-symbol PA DAML scheme, the

PA MLSD effectively improves the BER performance of the coherent receiver

In the linear phase noise channel, there is above 1dB SNR penalty between thesetwo schemes at BER= 10−4when laser linewidth is 500kHz

1.4 Outline of the Thesis

First of all, in Chapter 2, we briefly review some background knowledge inthe coherent communication system Advanced modulation formats togetherwith their generation methods using the Mach-Zehnder modulators (MZM) areintroduced Various channel distortions of optical fiber link such as CD, PMD,fiber nonlinearities, and their corresponding impacts on transmitted signals aredescribed in detail In addition, the structure of the coherent receiver and thereceived signal model are explained The off-line DSP blocks applied in thecoherent receiver for various purpose are also described in this chapter

The MLSD algorithm is theoretically derived in Chapter 3 The mentable metric function to choose survivor in each state of the Viterbi trellis

imple-is obtained from original conditional probability function Monte Carlo tions of M -PSK and M -QAM signals are carried out in linear phase noise chan-nel and long-haul transmission system to compare the BER performance of theproposed MLSD algorithm with other methods, especially data-aided schemessuch as DA ML PE The results show that MLSD effectively improves thelaser linewidth tolerance and transmission distance compared with symbol-by-symbol methods Trade-off between averaging additive noise, nonlinear phasenoise and tracking the linear phase noise accurately is observed Theoreticalanalysis is conducted to derive the optimal window length to achieve the opti-mal performance However, the optimal window length can only be obtainedwith the availability of channel statistics such as SNR, phase noise variance andtransmission distance, which is not practical in the real case

simula-To solve this problem, the adaptive version of MLSD is proposed in Chapter

4 to automatically deliver optimal performance without the requirement of nel statistics A first-order adaptive filter is applied to estimate the phase noise.The filter gain can be viewed as a forgetting factor which minimizes the sum

chan-of past estimation error Therefore, the whole past information can be utilized

to update the phase reference, which is the estimated carrier phase Numericalinvestigations are conducted through Monte Carlo simulations for both linearand nonlinear channels The results show that the adaptive MLSD outperformsthe non-adaptive version with reduced memory requirement in the coherent re-ceiver Experiments of 100-Gbit/s PDM DQPSK signals are also demonstrated

to verify our results

1.4 Outline of the Thesis

In Chapter 5, a pilot-assisted MLSD algorithm is introduced to mitigate thecycle slip effect, which is caused by decision errors, without the need of differ-ential encoding/decoding A small number of pilot symbols are inserted period-ically between adjacent data blocks to re-calculate the phase reference before it

is corrupted by the accumulated estimation errors Therefore, the carrier phaseestimation is always maintained on track Similarly, we first conducted MonteCarlo simulations in linear and nonlinear channels A symbol-by-symbol pilotassisted DA ML scheme is also studied for comparison In addition, 10-GBaud/s8-QAM experiments are carried out and we studied two popular 8-QAM for-mats: square 8-QAM and rotated 8-QAM In both schemes, our proposed PAMLSD algorithm outperforms conventional ones

Finally, we conclude this thesis in Chapter 6 Discussions on future studiesare also given in this chapter