mitra dsp c Signal Signal Processing

Chapter 1 Signals Signal Processing 清大電機系林嘉文 cwlinee.nthu.edu.tw 035731152 33120 © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 111 Signal Signal Processing • Signal: quantity that carries information • Signal Processing is to study how to represent, convert interpret and process a signal and the , interpret, and process a signal and the information contained in the signal • DSP i l i i th di it l d i : signal processing in the digital domain © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 112Signals Systems • Signals – “Something” that carries information – S h di i id bi di l i l peech, audio, image, video, biomedical signals, radar signals, seismic signals, etc. • S t ystems – “Something” that can manipulate, change, record, or transmit signals – Examples: CD, VCDDVD © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 113“DiscreteTime” Signals vs. “Digital” Signals • DiscreteTime signal – A “sampled” version of a continuous signal – Wh t h ld b th li f hi h i at should be the sampling frequency which is enough for perfectly reconstructing the original continuous signal? • Nyquist rate (Shannon sampling theorem) • Digital Signal – Sampling + Quantization – Quantization: use a number of finite bits (e g 8 .g., 8 bits) to represent a sampled value © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 114Examples of Typical Signals • Speech and music signals Represent air pressure as a function of time at a point in space • Waveform of the speech signal “ I like digital signal processing” : © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 115Digital Speech Signals • Voice frequency range: 20Hz 3 4 KHz ~ 3.4 KHz • Sampling rate: 8 KHz (8000 samplessec) • Quantization: 8 bitssample • Bitrate: 8K samp p lessec 8 bitssample = 64 Kbps (for uncompressed digital phone) • In current Voice over IP (VOIP) technology, digital speech signals are usually compressed (compression ratio: 8 10) ~10) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 116Examples of Typical Signals • Dow Jones Industrial Average © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 117Examples of Typical Signals • Electrocardiography (ECG) Signal Represents the electrical activity of heart © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 118ECG Signal • Th ECG t i i di f e ECG trace is a periodic waveform • One period of the waveform shown below represents one cycle of the blood transfer process from the heart to the arteries © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 119Examples of Typical Signals • Electroencephalogram (EEG) Signals Represent the electrical activity caused by the random firings of billions of neurons in the brain © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1110Examples of Typical Signals • Seismic Signals Caused by the movement of rocks resulting from an earthquake, a volcanic eruption or an underground explosion , or an underground explosion © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1111Examples of Typical Signals • Blackandwhite picture Represents light intensity as a function of two spatial coordinates I (x ,y) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1112Examples of Typical Signals • Color Image – Consists of Red Green and , Green, and Blue (RGB) components © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1113Examples of Typical Signals • Surface Search Radar Image © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1114Digital Image • An one megapixel image (1024x1024) • Quantization: 24 bitspixel for the RGB fullcolor sp , p ace, and 12 bitspixel for a reduced color space (YCbCr) • Bitrate: 1024x1024 samplessec 12 bitspixel = 12 Mbits = 1.5 Mbytes (for uncompressed digital phone) • How many uncompressed images can be stored in a 2G SD flashmemory card? • What is the compression ratio of JPEG used in di it l ? © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1115 your g a cameraDigital Image (Cont ) .) • In your image processing course, you were taught how to do – Edge detection (highpass filtering) – Image blurring or noise reduction (lowpass filtering) – Object segmentation (spatial coherence classification) – Image compression (retaining most significant info) • The above are all about mathematical manipulations – Could you give mathematical formulations for the above manipulations? – Could you characterize the frequency behaviors of the above operations? – Could you design an image processing tool to meet a © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1116 given spec?Example of Digital Image Processing Original Image Edge Detection © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1117 BlurringExamples of Typical Signals • Vid i l eo signals C i t f f onsists of a sequence of images, called frames, and is a function of 3 vari bl 2 ti l di t d ti ables: 2 spatial coordinates and time Frame 1 Frame 3 Frame 5 © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1118Classifications of Signals (14) • Types of signal depend on the nature of the i d d t i bl d th l f th f ti ndependent variables and the value of the function defining the signal – for example the independent variables can be continuous or , the independent variables can be continuous or discrete – likewise, the signal can be a continuous or discrete function of the independent variables – for an 1D signal, the independent variable is usually labeled as time • A signal can be either a realvalued function or a complexvalued function • A signal generated by a single source is called a scalar signal, where as a signal generated by multiple sources © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1119 is called a vector signal or a multichannel signalClassifications of Signals (24) • A continuoustime signal is defined at every instant of time • A discretetime signal is defined at discrete instants of time and hence it is a seq ence of n mbers , and hence, it is a sequence of numbers • A continuoustime signal with a continuous amplitude is usually called an analog signal (e g speech) .g., speech) • A discretetime signal with discretevalued amplitudes represented by a finite number of digits is referred to as a digital signal • A discretetime signal with continuousvalued amplitudes is called a sampleddata signal • A continuoustime signal with discretevalue amplitudes © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1120 is usually called a quantized boxcar signalClassifications of Signals (34) • A signal that can be uniquely determined by a welldefined process, such as a mathematical expression or rule or table look , or table lookup is called a , is called a deterministic signal • A signal that is generated in a random fashion and cannot be predicted ahead of time is called a random signal © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1121Classification of Signals (44) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1122Typical Signal Processing Operations • Most signal processing operations in the case of analog signals are carried out in the timedomain • In the case of discretetime signals, both timedomain or frequencydomain operations are usually employed • Continuoustime Fourier transform (CTFT) is used to transform a signal into the frequency domain © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1123Elementary TimeDomain Operations • Three most basic timedomain signal operations: scaling, delay, and addition • Integration • Differentiation • More complex operations are implemented by combining two or more elementary operations © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1124Filtering (13) • Filtering is one of the most widely used complex signal processing operations • A filter passes certain frequency components and blocks other frequency components • Passband vs. stopband of a filter • The filtering operation of a linear analog filter is described by the convolution integral where x(t) is the input signal, y(t) is the output of the filter, and h(t) is the impulse response of the filter © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1125Filtering (23) • Frequencyselective filters can be classified into the following types according to their passbands and stopbands: lowpass, highpass, bandpass, and bandstop filters • Notch filter: blocks a single frequency component • Multiband filter: has more than one passband and more than one stopband • Comb filter: blocks frequencies that are integral multi l f l f ples of a low frequency © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1126Filtering (33) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1127Modulation • For efficient transmission of a lowfrequency signal over a channel, it is necessary to transform the signal to a highfrequency signal by means of a modulation operation • Four major types of modulation of analog signals: – Amplitude modulation – Frequency modulation – Phase modulation – Pulse amplitude modulation © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1228Amplitude Modulation (13) • The a p tude o a g mplitude of a highfreque cy s uso da s g a ncy sinusoidal signal Acos(Ωot), called the carrier signal, is varied by a lowfrequency signal x(t), called the modulating signal by upper sideband lower sideband © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1229 DoubleSideBand Suppressed Carrier (DSBSC) modulationAmplitude Modulation (23) • To demodulate, y(t) is first multiplied with a sinusoidal signal of the same frequency as the carrier: • Thus x(t) can be recovered from r(t) by passing it through a lowpass filter with a cutoff frequency at Ωc satisfying th l ti e relation Ω Ω 2Ω Ω m < c < o − m © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1230Amplitude Modulation (33) • Modulation Demodulation of AM: • A modulating signal (20 Hz) and the amplitudemodulated carrier (400 Hz) obtained using the DSB modulation © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1231Hilbert Transform • The impu se lse respo se nse of Hilbe t rt t a s o ransform is de ed fined as • The continuoustime F i ourier t f ransform (CTFT) XHT(jΩ) of h HT(t) is • The input signal x(t) can be divided into two components: X ( jΩ) = X p( jΩ) + X n( jΩ) where X p(jΩ) is the portion of X(jΩ) occupying the positive frequency range and Xn(jΩ) is the portion i th ti f © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1232 occupy ng e nega ve requency rangeHilbert Transform (22) • The CTFT of y(t) becomes • Consider g(t) = x(t) + j y(t). The CTFT of g(t) is only the positive frequency component is retained © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1233 SingleSideBand (SSB) Modulation © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1234Quadrature Amplitude Modulation (13) • QAM uses DSB modulation to modulate two different signals so that they both occupy the same bandwidth • The two carrier signals have the same carrier frequency Ω o but have a phase difference of 90o • QAM takes up as much bandwidth as the SSB method, and only half of DSB © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1235Quadrature Amplitude Modulation (23) • To recover x1(t) and x2(t), y(t) is multiplied by both the inphase and the quadrature components of the carrier sep y arately: • Lowpass filtering of r1(t) and r2(t) by filters with a cutoff at Ω m yields x1(t) and x2(t) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1236 Quadrature Amplitude Modulation (33) • QAM Modulation Demodulation： © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1237Multiplexing Demultiplexing • Purpose: For an efficient utilization of a wideband channel, many narrowbandwidth lowfrequency signals are combined for a composite wideband signal that is t itt d i l i l ransmitted as a single signal • Illustration of FrequencyDivision Multiplexing (FDM): © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1238Why DSP? • Mathematical abstractions lead to generalization and discovery of new processing techniques • Computer implementations are flexible • Applications provide a physical context © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1339Advantages of DSP (12) • Absence of drift in the filter characteristics – Processing characteristics are fixed, e.g. by binary coefficients stored in memories – Independent of the external environment and of p p g g arameters such as temperature and device aging • Improved quality level – Quality of processing limited only by economic considerations – Desired q y y g uality level achieved by increasing the number of bits in datacoefficient representation (SNR improvement: 6 dBbit) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1340Advantages of DSP (22) • Reproducibility – Component tolerances do not affect system performance with correct operation – No adjustments necessary during fabrication – No realignment needed over lifetime of equipment • Ease adjustment of processor characteristics – Easy to develop and implement adaptive filters, programmabl filt d l t filt e filters and complementary filters • Timesharing of processor (multiplexing modularity) • No loading effect • Realization of certain characteristics not possible or diffi lt ith l i l t ti © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1341 cu w ana og mp emen a onsLimitations of DSP • Limited Frequency Range of Operation – Frequency range technologically limited to values corresponding to maximum computing capacities (e.g., AD converter) that can be developed and exploited • Digital systems are active devices, thereby consuming more power and being less reliable • Additional Complexity in the Processing of Analog Signals – AD and DA converters must be introduced adding complexity to overall system • Inaccuracy due to finite precision arithmetic © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1342 Application Examples of DSP • Cellular Phone • Discrete Multitone Transmission ( ) ADSL) • Digital Camera • Digital Sound Synthesis • Signal Coding Compression • Signal Enhancement © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1343Cellular Phone Block Diagram © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1344Cellular Phone Baseband SOC © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1345Discrete MultiTone Modulation (DMT) • Core technology in the implementation of the asymmetric digital subscriber line (ADSL) and veryhighrate DSL (VDSL) • ADSL: – Downstream bitrate: up to 9 Mbs – Upstream bitrate: up to 1 Mbs • VDSL: – Downstream bitrate: 13 to 26 Mbs – Upstream bitrate: 2 to 3 Mbs – Distance: less than 1 km • Orthogonal FrequencyDivision Multiplexing (OFDM) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1346 for wireless communicationsDMT Transmitter Receiver © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1347ADSL Band Allocation • Bandallocations for an ADSL system © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1348Digital Camera • CMOS Imaging Sensor – Increasing y g g ly being used in digital cameras – Single chip integration of sensor and other image p g g g g rocessing algorithms needed to generate final image – Can be manufactured at low cost – Less expensive cameras use single sensor with individual pixels in the sensor covered with either a red, a green, or a blue optical filter © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1349Digital Camera • Image Processing Algorithms – Bad pixel detection and masking – Color interpolation – Color balancing – Contrast enhancement – False color detection and masking – Image and video compression © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1350Digital Camera • Bad pixel detection and masking © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1351Digital Camera • Color Interpolation and Balancing © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1352Digital Sound Synthesis (15) • Four methods for the synthesis of musical sound: – Wavetable synthesis – Spectral modeling synthesis – Nonlinear synthesis – Synthesis by physical modeling © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1353Digital Sound Synthesis (25) • W t bl S th i avetable Synthesis – Recorded or synthesized musical events stored in internal memory and played back on demand – Playback tools consists of various techniques for sound variation during reproduction such as pitch shifting, looping, enveloping and filtering – Example： Giga sampler © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1354Digital Sound Synthesis (35) • Spectral Modeling Synthesis – Produces sounds from freq y uency domain models – Signal represented as a superposition of basis functions with timevary g p ing amplitudes – Practical implementation usually consist of a combination of additive synthesis, subtractive synthesi,s and granular synthesis – Example: Kawaii K500 Demo © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1355Digital Sound Synthesis (45) • N li S th i onlinear Synthesis – Frequency modulation method: Time dependent phase terms in the sinusoidal basis Functions – An inexpensive method frequently used in synthesizers and in sound cards for PC – Example: Variation modulation index complex algorithm (Pulsar) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1356Digital Sound Synthesis (55) • Physical Modeling – Models the sound production method – Physical description of the main vibrating structures by partial differential equations – Physical description of the main vibrating structures by partial differential equation – Examples: (CCRMA, Stanford) • Guitar with nylon strings • Marimba（ 木琴） • Tenor saxophone © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1357Signal Coding Compression • Concerned with efficient digital representation of audio or visual signal for storage and transmission to provide maximum quality to the listener or viewer © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1358Signal Compression Example (13) • Original Speech Data size: 330,780 bytes • Compressed Speech(GSM 6 10) .10) Sampled at 22.050 kHz, Data size 16,896 bytes • Compressed speech (Lernout Hauspie CELP 4 8kbits) .8kbits) Sampled at 8 kHz, Data size 2,302 bytes © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1359Signal Compression Example (23) • Original Music Audio Format: PCM 16.000 kHz, 16 Bits (Data size 66206 bytes) • Compressed Music Audio Format: GSM 6.10, 22.05 kHz (Data size 9295 bytes) Courtesy: Dr. A. Spanias © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1360Signal Compression Example (33) Original Lena Image Fil Si 256K b t Compressed Lena Image Fil Si 13K b t © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1361 e ze = y es e ze = y esApplications: Signal Enhancement • Purpose: To emphasize specific signal features to provide maximum quality to the listener or viewer • For speech sig g nals, algorithms include removal of background noise or interference • For image or video signals, algorithms include contrast enhancement, sharpening and noise removal © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1362Signal Enhancement Examples (14) • Noisy speech signal (10% impulse noise) • Noise removed speech © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1363Signal Enhancement Examples (24) © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1364Signal Enhancement Examples (34) • Original image and its contrast enhanced version Original Enhanced © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1365Signal Enhancement Examples (44) • Noisy image denoised image © The McGrawHill Companies, Inc., 2007 Original PowerPoint slides prepared by S. K. Mitra 1366

Chapter 1 Chapter 1

Signals & Signal Processing

清大電機系林嘉文 cwlin@ee.nthu.edu.tw

03-5731152 #33120 03-5731152 #33120

Signal & Signal Processing

• Signal: quantity that carries information Signal: quantity that carries information

• Signal Processing is to study how to represent, convert interpret and process a signal and the

convert, interpret, and process a signal and the information contained in the signal

DSP i l i i th di it l d i

• DSP: signal processing in the digital domain

1-1-2

Signals & Systems

“Discrete-Time” Signals vs “Digital”

Signals

• Discrete Time signal

– A “sampled” version of a continuous signal

Wh t h ld b th li f hi h i

– What should be the sampling frequency which is enough for perfectly reconstructing the original continuous signal?

• Nyquist rate (Shannon sampling theorem)

• Digital Signal

– Sampling + QuantizationQuantization: use a number of finite bits (e g 8

– Quantization: use a number of finite bits (e.g., 8 bits) to represent a sampled value

1-1-4

Examples of Typical Signals

• Speech and music signals - Represent air

pressure as a function of time at a point in space

• Waveform of the speech signal “ I like digital signal processing” : g g

Digital Speech Signals

• Voice frequency range: 20Hz 3 4 KHz

• Voice frequency range: 20Hz ~ 3.4 KHz

• Sampling rate: 8 KHz (8000 samples/sec)

• Quantization: 8 bits/sample

• Bit-rate: 8K samples/sec * 8 bits/sample = 64 p p

Kbps (for uncompressed digital phone)

• In current Voice over IP (VOIP) technology In current Voice over IP (VOIP) technology,

digital speech signals are usually compressed (compression ratio: 8~10)

1-1-6

Examples of Typical Signals

• Dow Jones Industrial Average

Examples of Typical Signals

• Electrocardiography (ECG) Signal - Represents

the electrical activity of heart

1-1-8

ECG Signal

Th ECG t i i di f

• The ECG trace is a periodic waveform

• One period of the waveform shown below

represents one cycle of the blood transfer process from the heart to the arteries

Examples of Typical Signals

• Electroencephalogram (EEG) Signals

-Represent the electrical activity caused by the

random firings of billions of neurons in the brain

1-1-10

Examples of Typical Signals

• Seismic Signals Caused by the movement

• Seismic Signals - Caused by the movement

of rocks resulting from an earthquake, a volcanic eruption or an underground explosion

Examples of Typical Signals

• Black-and-white picture - Represents light

intensity as a function of two spatial coordinates

I (x ,y)

1-1-12

Examples of Typical Signals

• Color Image – Consists of Red Green and Color Image Consists of Red, Green, and

Blue (RGB) components

Examples of Typical Signals

• Surface Search Radar Image Surface Search Radar Image

1-1-14

Digital Image

• An one mega-pixel image (1024x1024)

• Quantization: 24 bits/pixel for the RGB full-color space, and 12 bits/pixel for a reduced color p , p

space (YCbCr)

• Bit-rate: 1024x1024 samples/sec * 12 bits/pixel Bit rate: 1024x1024 samples/sec 12 bits/pixel

= 12 Mbits = 1.5 Mbytes (for uncompressed digital phone)

• How many uncompressed images can be

stored in a 2G SD flash memory card?

• What is the compression ratio of JPEG used in

your digital camera?

Digital Image (Cont )

• In your image processing course, you were taught how

to do– Edge detection (high-pass filtering)– Image blurring or noise reduction (low-pass filtering)Image blurring or noise reduction (low-pass filtering)– Object segmentation (spatial coherence classification)– Image compression (retaining most significant info)

• The above are all about mathematical manipulations

Could you give mathematical formulations for the

– Could you give mathematical formulations for the above manipulations?

– Could you characterize the frequency behaviors ofCould you characterize the frequency behaviors of the above operations?

– Could you design an image processing tool to meet a

1-1-16

Could you design an image processing tool to meet a given spec?

Example of Digital Image Processing

Edge Detection Original Image

Blurring

Examples of Typical Signals

• Video signals - Consists of a sequence of

images, called frames, and is a function of 3

i bl 2 ti l di t d ti variables: 2 spatial coordinates and time

1-1-18

Classifications of Signals (1/4)

Classifications of Signals (1/4)

• Types of signal depend on the nature of the

i d d t i bl d th l f th f ti

independent variables and the value of the function

defining the signal

for example the independent variables can be continuous or

– for example, the independent variables can be continuous or discrete

– likewise, the signal can be a continuous or discrete function of the independent variables

– for an 1-D signal, the independent variable is usually labeled as time

• A signal can be either a real-valued function or a

complex-valued functionp

• A signal generated by a single source is called a scalar

is called a vector signal or a multichannel signal

Classifications of Signals (2/4)

Classifications of Signals (2/4)

titime

time and hence it is a seq ence of n mbers

• A continuous-time signal with a continuous amplitude is

usually called an analog signal (e g speech)

usually called an analog signal (e.g., speech)

• A discrete-time signal with discrete-valued amplitudes

represented by a finite number of digits is referred to as

represented by a finite number of digits is referred to as

• A discrete-time signal with continuous-valued amplitudesA discrete time signal with continuous valued amplitudes

is called a sampled-data signal

• A continuous-time signal with discrete-value amplitudes

1-1-20

is usually called a quantized boxcar signal

Classifications of Signals (3/4) Classifications of Signals (3/4)

• A signal that can be uniquely determined by a

well-defined process, such as a mathematical expression or

rule or table look up is called a deterministic signal

rule, or table look-up, is called a deterministic signal

• A signal that is generated in a random fashion and

cannot be predicted ahead of time is called a random

cannot be predicted ahead of time is called a random

signal

Classification of Signals (4/4)

1-1-22

Typical Signal Processing

Operations

• Most signal processing operations in the case of analog signals are carried out in the time-

domain

• In the case of discrete-time signals, both time- g

domain or frequency-domain operations are

usually employed

• Continuous-time Fourier transform (CTFT) is

used to transform a signal into the frequency

used to transform a signal into the frequency

domain

Elementary Time Domain Operations

• Three most basic time-domain signal operations: scaling, delay, and addition

• Integration

• Differentiation

• More complex operations are implemented by

combining two or more elementary operations

1-1-24

Filtering (1/3)

• Filtering is one of the most widely used complex signal

• Filtering is one of the most widely used complex signal

processing operations

• A filter passes certain frequency components andA filter passes certain frequency components and

blocks other frequency components

• Passband vs stopband of a filterp

• The filtering operation of a linear analog filter is

described by the convolution integral

where x(t) is the input signal, y(t) is the output of the filter, and h(t) is the impulse response of the filter

Filtering (2/3)

• Frequency-selective filters can be classified into the

following types according to their passbands and

stopbands: low pass high pass bandpass and

stopbands: low-pass, high-pass, bandpass, and

bandstop filters

• Notch filter: blocks a single frequency component

• Notch filter: blocks a single frequency component

• Multiband filter: has more than one passband and

more than one stopband

• Comb filter: blocks frequencies that are integral

lti l f l fmultiples of a low frequency

1-1-26

Filtering (3/3)

• For efficient transmission of a low-frequency signal over

a channel, it is necessary to transform the signal to a

high frequency signal by means of a modulation

high-frequency signal by means of a modulation

operation

• Four major types of modulation of analog signals:Four major types of modulation of analog signals:

– Amplitude modulation– Frequency modulation– Phase modulation

Pulse amplitude modulation– Pulse amplitude modulation

1-2-28

Amplitude Modulation (1/3)

• The amplitude of a high-frequency sinusoidal signal e a p tude o a g eque cy s uso da s g a

Acos(Ω o t), called the carrier signal, is varied by a

low-frequency signal x(t), called the modulating signal by

upper sideband

lower sideband

Double-SideBand Suppressed Carrier (DSB-SC) modulation

Amplitude Modulation (2/3)

• To demodulate y(t) is first multiplied with a sinusoidal

• To demodulate, y(t) is first multiplied with a sinusoidal

signal of the same frequency as the carrier:

• Thus x(t) can be recovered from r(t) by passing it through

a low-pass filter with a cutoff frequency at Ωc satisfying

th l ti Ω Ω 2Ω Ω

the relation Ωm < Ω c < 2Ω o −Ω m

1-2-30

Amplitude Modulation (3/3)

• Modulation & Demodulation of AM:

• A modulating signal (20 Hz) and the amplitude-modulated carrier (400 Hz) obtained using the DSB modulation

) (

) ( j Ω = X j Ω + X j Ω

where X p (jΩ) is the portion of X(jΩ) occupying the

positive frequency range and X n (jΩ) is the portion

1-2-32

occupying the negative frequency range

Hilbert Transform (2/2)

• The CTFT of y(t) becomes The CTFT of y(t) becomes

• Consider g(t) = x(t) + j y(t) The CTFT of g(t) is

only the positive frequency component is retained

only the positive-frequency component is retained

Single SideBand (SSB) Modulation

1-2-34

Quadrature Amplitude Modulation (1/3) Quadrature Amplitude Modulation (1/3)

• QAM uses DSB modulation to modulate two different

• QAM uses DSB modulation to modulate two different

signals so that they both occupy the same bandwidth

• The two carrier signals have the same carrier frequency

• The two carrier signals have the same carrier frequency

Ωo but have a phase difference of 90o

QAM takes up as much bandwidth as the SSB method

• QAM takes up as much bandwidth as the SSB method,

and only half of DSB

Quadrature Amplitude Modulation (2/3) Quadrature Amplitude Modulation (2/3)

• To recover x (t) and x (t) y(t) is multiplied by both the

in-• To recover x1(t) and x2(t), y(t) is multiplied by both the

in-phase and the quadrature components of the carrier

Quadrature Amplitude Modulation (3/3) Quadrature Amplitude Modulation (3/3)

• QAM Modulation & Demodulation：

Multiplexing & Demultiplexing

• Purpose: For an efficient utilization of a wideband

channel, many narrow-bandwidth low-frequency signals

are combined for a composite wideband signal that is

t itt d i l i l

transmitted as a single signal

• Illustration of Frequency-Division Multiplexing (FDM):

1-2-38

Why DSP?

• Mathematical abstractions lead to Mathematical abstractions lead to

generalization and discovery of new processing

techniques

• Computer implementations are flexible

• Applications provide a physical context

Advantages of DSP (1/2)

• Absence of drift in the filter characteristics Absence of drift in the filter characteristics

– Processing characteristics are fixed, e.g by binary coefficients stored in memories

coefficients stored in memories – Independent of the external environment and of parameters such as temperature and device aging

• Improved quality level

– Quality of processing limited only by economicQuality of processing limited only by economic considerations

– Desired quality level achieved by increasing the q y y gnumber of bits in data/coefficient representation (SNR improvement: 6 dB/bit)

1-3-40

• Ease adjustment of processor characteristics

– Easy to develop and implement adaptive filters,

bl filt d l t filt

programmable filters and complementary filters

• Time-sharing of processor (multiplexing & modularity)

Limitations of DSP

• Limited Frequency Range of Operation Limited Frequency Range of Operation

– Frequency range technologically limited to values corresponding to maximum computing capacities (e.g.,

corresponding to maximum computing capacities (e.g., A/D converter) that can be developed and exploited

• Digital systems are active devices, thereby Digital systems are active devices, thereby

consuming more power and being less reliable

• Additional Complexity in the Processing of Analog

• Additional Complexity in the Processing of Analog Signals

A/D and D/A converters must be introduced adding

– A/D and D/A converters must be introduced adding complexity to overall system

• Inaccuracy due to finite precision arithmetic

1-3-42

• Inaccuracy due to finite precision arithmetic

Application Examples of DSP

• Cellular Phone

• Discrete Multitone Transmission (ADSL) ( )

• Digital Camera

• Digital Sound Synthesis

• Signal Coding & Compression

• Signal Enhancement

Cellular Phone Block Diagram

1-3-44

Cellular Phone Baseband SOC

Discrete MultiTone Modulation (DMT)

• Core technology in the implementation of the

asymmetric digital subscriber line (ADSL) and high-rate DSL (VDSL)

very-• ADSL:

– Downstream bit-rate: up to 9 Mb/s– Upstream bit-rate: up to 1 Mb/s

• VDSL:

– Downstream bit-rate: 13 to 26 Mb/s– Upstream bit-rate: 2 to 3 Mb/s

– Distance: less than 1 km

• Orthogonal Frequency-Division Multiplexing (OFDM)

1-3-46

for wireless communications

Transmitter

Receiver

ADSL Band Allocation

• Band-allocations for an ADSL system

1-3-48

Digital Camera

• CMOS Imaging Sensor

– Increasingly being used in digital camerasg y g g– Single chip integration of sensor and other image processing algorithms needed to generate final image

Digital Camera

• Image Processing Algorithms

– Bad pixel detection and masking– Color interpolation

– Color balancing– Contrast enhancement– False color detection and masking– Image and video compression

1-3-50

Digital Camera

• Bad pixel detection and masking

Digital Camera

• Color Interpolation and Balancing

1-3-52

Digital Sound Synthesis (1/5)

• Four methods for the synthesis of musical sound:

– Wavetable synthesis– Spectral modeling synthesis– Nonlinear synthesis

– Synthesis by physical modeling

Digital Sound Synthesis (2/5)

W t bl S th i

• Wavetable Synthesis

– Recorded or synthesized musical events stored in internal memory and played back on demand

– Playback tools consists of various techniques for

sound variation during reproduction such as pitch shifting, looping, enveloping and filtering

– Example：Giga sampler

1-3-54

Digital Sound Synthesis (3/5)

• Spectral Modeling Synthesis

– Produces sounds from frequency domain modelsq y

– Signal represented as a superposition of basis

functions with time-varying amplitudesy g p– Practical implementation usually consist of a

combination of additive synthesis, subtractive

combination of additive synthesis, subtractive synthesi,s and granular synthesis

– Example: Kawaii K500 DemoExample: Kawaii K500 Demo

Digital Sound Synthesis (4/5)

Digital Sound Synthesis (5/5)

– Physical description of the main vibrating structures

by partial differential equationsPhysical description of the main vibrating structures

– Physical description of the main vibrating structures

by partial differential equation– Examples: (CCRMA Stanford)Examples: (CCRMA, Stanford)

• Guitar with nylon strings

• Marimba（木琴）

• Tenor saxophone