Digital Video Compression Fundamentals and Standards docx

Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions... Introduction Video Compression Standards Simulation Reference Software Future Work a

Digital Video Compression Fundamentals and Standards

Speaker: Wei-Yi Wei Advisor: Prof Jian-Jung Ding

Introduction

Video Compression Standards

Simulation Reference Software

Future Work and Conclusions

Introduction

Video Compression Standards

Simulation Reference Software

Future Work and Conclusions

Introduction (1/2)

Why video compression technique is important ?

One movie video without compression

720 x 480 pixels per frame

30 frames per second

Total 90 minutes

Full color

The most intuitive method to remove

Spatiotemporal redundancy

3-Dimensional DCT

Remove spatiotemporal correlation

Good for low motion video

Bad for high motion video

The most popular method to remove

temporal redundancy

The Block-Matching Algorithm

Matching Function

The dissimilarity between two blocks and

The matching criteria

Mean square error (MSE)

High precision is needed

Mean absolute difference (MAD)

The Exhaustive Block-Matching

Algorithm

Reference Frame Current Frame

Motion Vector

Fast Block-Matching Algorithms

EBMA needs Intensive computation

Fast Algorithm is needed

Find the possible local optimal

Fast Block-Matching Algorithms

The characteristics of fast algorithm

Not accurate as EBMA

Save large computation

Two famous fast algorithm

2-D logarithm Search Method

Three Steps Search Method

2-D logarithm Search Method

1 1

2

2

3 4

4 4

1

4

Three Step Search Method

1 1

1 1

Multiresolution Motion Estimation (1/3)

The number of levels is L

l -th level images of the target frames

where is set of pixels at level L

At the l -th level, the MV is

At the l -th level, the estimated MV is

Determine update such that error is minimized

The new motion vector is

Multiresolution Motion Estimation (2/3)

Variable block size method

Multiresolution Motion Estimation (3/3)

Introduction

Video Compression Standards

Simulation Reference Software

Future Work and Conclusions

The Development of Video

Compression Standards

The MPEG-1 Standard

Group of Pictures (1/2)

I-frame (Intracoded Frame)

Coded in one frame such as DCT

This type of frame do not need previous frame

P-frame (Predictive Frame)

One directional motion prediction from a previous frame

The reference can be either I-frame or P-frame

Generally referred to as inter-frame

B-frame (Bi-directional predictive frame)

Bi-directional motion prediction from a previous or future frame

The MPEG-1 Encoder (1/4)

Motion Estimation

Motion Estimation

Intra-frame

Entropy Coding

Residue

Motion Vector

Inter-frame

The MPEG-1 Encoder (2/4)

Differential Coding

is the input image

is the predictive image

The MPEG-1 Encoder (3/4)

The MPEG-1 Encoder (4/4)

Motion Compensation

Exploit motion vector and the previous reconstructed frame to generate the predictive frame

is the compensated image

is the previous image

is the motion vector

The MPEG-2 Standard

Field/Frame DCT Coding

Field/Frame Prediction Mode Selection

Alternative Scan Order

Various Picture Sampling Formats

User Defined Quantization Matrix

Progressive Scan and Interlaced

Scan

Field/Frame DCT Coding

The field type DCT

Fast motion video

The frame type DCT

Slow motion video

Field DCT Coding Luminance MB Frame DCT Coding

Alternative Scan Order

Zigzag scan order

0 0 0 0 0 0 0 0 0.85 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

2D DCT

The MPEG-2 Encoder (2/2)

Quantization

User can change the quantization if necessary

Intra quantization matrix

Inter quantization matrix

Various picture sampling formats

The MPEG-2 Encoder (1/2)

+

Q -1

Bits Enhance SNR Enhanced Layer

+

Base Layer Bits Base

Variable Block Size

Multiple Reference Frames

Integer Transform

Intra Prediction

In-loop Deblocking Filtering

1/4-pel Resolution Motion Estimation

CAVLC

Variable Block Size

The fixed block size may not be suitable for all

motion objects

Improve the flexibility of comparison

Reduce the error of comparison

7 types of blocks for selection

0 1 1

16 x 16 16 x 8 8 x 16 8 x 8

8 x 8 8 x 4 4 x 8 4 x 4

Multiple Reference Frames

The neighboring frames are not the most similar in some cases The B-frame can be reference frame

B-frame is close to the target frame in many situations

Integer Transform for Reducing The

Spatial Redundancy (1/2)

The transform matrix C

4×4 Block Size

Separable Integer Transform

The transform coefficients are CXCT

Integer Transform for Reducing The

Intra Prediction

Predict the similarity between the neighboring pixels in one frame in advance, and exploit differential coding transform coding to remove the redundancy

Vertical

Horizontal + + + +

+ + + +

Transform/

Quantization

Entropy Coding

Remove Perceptual Redundancy

In-loop deblocking filtering

Remove blocking artifact

Result from block based motion compensation

Result from block based transform coding

q0 q1 q2 q3

QP

1/4-pel Resolution Motion

The H.264/AVC Encoder

Transform/

Quantization

Entropy Coding

Inverse Transform/

De-Quantization Motion

Compensation

De-blocking Filter Motion

Residue

Motion Vector

Intra-frame Prediction

Coder

Controller

Control Data

Introduction

Video Compression Standards

Simulation Reference Software

Future Work and Conclusions

Reference Software Demo

Introduction

Video Compression Standards

Simulation Reference Software

Future Work and Conclusions

Future Work

Fast Mode Decision Algorithm

Interpolation Filter Design

Deblocking Filter Design

DCT-Based Motion Estimation

Implementation Based on TI DSP

Fast Mode Decision Algorithm

The computational cost of H.264 is large

Variable block-size ME

Multiple reference frames

Fast mode decision is needed for reducing the computation

time

Interpolation Filter Design

In order to estimate and compensate the fractional-pel

displacements

Adaptive Interpolation filter for replacing the fixed coefficient filter

Deblocking Filter Design

Block based ME and Transform result in the annoying

blocking artifact

Reduce the blocking artifact can increase the quality of

compressed video

DCT Based Motion Estimation

Robust even in noisy environment

Implementation Based on TI DSP

TMS320C6416

TI DM642

Thank You

Q & A

References (1/2)

[1] Yun Q.Shi and Huifang Sun, “Image and Video Compression for Multimedia

Engineering: Fundamentals, Algorithms, and Standards”, CRC press, 2000.

[2] Yao Wand, Jorn Ostermann and Ya-Qin Zhang, “Video Processing and

Communications”, Prentice Hall, 2007.

[3] Richardson, Lain E G., “Video Codec Design: Developing Image and Video

Compression Systems”, John Wiley & Sons Inc, 2002.

[4] Barry G, Haskell, Atul Puri and Arun N Netravali, “Digital Video : An

Introduction to MPEG-2”, Boston : Kluwer Academic, 1999.

[5] T Wiegand, G J Sullivan, G Bjontegaard, and A Luthra, “Overview of the

H.264/AVC video coding standard”, IEEE Trans on Circuits and systems for

video Technology, vol 13, no 7, pp 560-576, July 2003.

[6] G Sullivan and T Wiegand, “Video Compression - From Concepts to the

H.264/AVC Standard”, Proceedings of the IEEE, Special Issue on Advances in

Video Coding and Delivery, December 2004

華， 2004

References (2/2)

[8] Thomas Wedi, “Adaptive Interpolation Filters and High-Resolution

Displacements For Video Coding”, IEEE Trans on Circuits and Systems For

Video Technology, vol.

[9] Dong-Hwan Kim, Hwa-Yong Oh, O˘guzhan Urhan, Sarp Ertürk and Gyu Chang, “Optimal Post-Process/In-Loop Filtering for Improved Video

Tae-Compression Performance”, IEEE Trans on Consumer Electronics, vol 53, no

4, Nov 2007

[10] Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, ” Fast Multi-Frame Motion Estimation for H.264 and Its Applications to Complexity-Aware Streaming”,

[11] Kai-Ting Cheng and Hsueh-Ming Hang, “Acceleration and Implementation

of H.264 Encoder and Scalable Extension of H.264 Decoder on TI DSP

Platform”, master thesis, June 2007