Complexity Controlled Side Information Creation for Distributed Scalable Video Coding45013

Complexity Controlled Side Information Creation for Distributed Scalable Video Coding Quang Hoang Van1, Le Dao Thi Hue2, Vien Dinh Du1, Vu Nguyen Hong3, and Xiem HoangVan2 1Hanoi Unive

Complexity Controlled Side Information Creation

for Distributed Scalable Video Coding

Quang Hoang Van1, Le Dao Thi Hue2, Vien Dinh Du1, Vu Nguyen Hong3, and Xiem HoangVan2

1Hanoi University of Industry

2VNU-University of Engineering and Technology

3Radio Electronics Association of Vietnam

quanghvdt@gmail.com, xiemhoang@vnu.edu.vn

Abstract—Distributed scalable video coding (DSVC)

has recently been gaining many attentions due to its

benefits in terms of computational complexity, error

resilience and scalability, which are important for

emerging video applications like wireless sensor

networks and visual surveillance system (VSS) In

DSVC, the side information (SI) creation plays a key

role as it directly affects the DSVC compression

performance and the encoder/decoder computational

complexity However, for many VSS applications, the

energy of each VSS node is usually attenuating along

the time, making the difficulty in transmitting

surveillance video in real time To address this problem,

we propose a complexity controlled SI creation solution

for the newly DSVC framework To achieve the flexible

SI creation, the complexity associated with SI creation

process is modeled using a linear model in which the

model parameters are estimated from a fitting process

To adjust the SI complexity, a user parameter is defined

based on the availability of the VSS energy resource

Experiments conducted for a rich set of video

surveillance data have revealed the benefits of the

proposed complexity control solution, notably in both

complexity control and compression performance

Keywords—Distributed scalable video coding, side

information, visual sensor networks

Nowadays, video surveillance systems (VSS)

have been widely used in many important

applications such as public safety and private

protection [1] Such a system can provide real-time

monitoring and analysis of the observed environment

Real-world video surveillance applications, which

typically require storing videos without neglecting

any part of scenarios for weeks or months In

addition, the heterogeneity of devices, networks and

environments is also gaining a request for adaptation

solutions In this scenario, there is a critical need of a

powerful video coding scheme that is featured by

high coding efficiency, scalability and low encoding

complexity capabilities

A VSS typically includes three main parts, the

camera nodes, the center and the users as shown in

Fig 1 The video is firstly captured and processed at

the camera node and sent to the server Such video

bitstream can be transcoded or distributed to users

with different quality, resolutions At the user side,

video data can be used for object detection, activity tracking, and/or event analysis

Camera 1

Camera 3 Camera 2

Camera 4

Server

Router

Local PC

Internet Portable

Decives

Remote PC

Monitor TV

Fig 1 A video surveillance system The recent researches have shown that the distributed scalable video coding [2-5], a newly High Efficiency Video Coding (HEVC) [6] scalable extension [6] can satisfy the mentioned requirements

of a VSS [7] However, for many VSS applications, the energy allowed at each VSS node is usually attenuating along the time In this case, the complexity

of DSVC should be adjusted depending on the energy situation in each VSS node In DSVC, the SI creation [8] usually consumes the largest percentage of computational complexity [2] In this context, we propose in this paper a novel complexity controlled SI creation solution to adaptively adjust the overall DSVC complexity, notably by a SI complexity-modeling framework

In the proposed SI creation solution, the complexity associated with the motion estimation stage is controlled using a user setting parameter Depending on the energy situation of each VSS node, the user parameter is imported to control the complexity of the SI creation process and thus the overall DSVC solution Experiments conducted for a rich set of test surveillance video has shown that the proposed SI creation solution is easy to manage the complexity at both the encoder and decoder

SHVC Intra Encoder

Sequence

Splitting

HEVC Intra Encoder

Key frames

WZ frames

Base Layer

Enhancement Layer

ˆ

B X

HEVC Intra Decoder

SHVC Intra Decoder

+

SI Residue Creation

Correlation Modeling

Sequence Merging Reconstructed EL

Reconstructed BL ˆ

B X

Syndrome Reconstruction

Correlation Modeling

Surveillance Videos

ˆ

B X

LSB

Syndrome Creation

Syndrome Decoding

SI Residue Creation

Fig 2 Distributed scalable video coding architecture (highlighting the SI creation component) The organization of this paper is as follows

Section II briefly introduces the background work on

DSVC Section III presents the proposed complexity

controlled SI creation while Section IV evaluates and

discusses the performance of the method Finally, in

Section V, some concluding remarks and future works

are outlined

II OVERVIEW OF DISTRIBUTED SCALABLE VIDEO

CODING Distributed scalable video coding (DSVC), was

firstly proposed by X HoangVan et al, in [2-5] The

DSVC is in fact an HEVC scalable extension, which

follows a layered coding approach with one base

layer (BL) and one or several enhancement layers

(ELs) Fig 2 illustrates the general architecture of the

DSVC in which two layers are presented In DSVC,

while the BL is backward compatible with the HEVC

standard [6], the EL is processed with the distributed

coding structure to achieve the low encoding

complexity and error resilience features [2]

Since the DSVC EL is compressed with the

distributed source coding approach [9], instead of

finding the best prediction which usually requires a

large number of computations, the EL compresses

only a part of original information (called syndrome)

which cannot be inferred at the decoder In such a

system, the correlation between the original and side

information is estimated at both the encoder and

decoder sides [3] Hence, the SI creation and the

correlation estimation model are critical parts in

DSVC The detail of the DSVC working steps can be

referred to [2]

Following the DSVC solution proposed in [2-5],

the SI creation is performed at both encoder and

decoder sides While the encoder SI is used to

estimate the correlation model, represented through a

number of least significant bitplanes (nLSB), the

decoder SI is used for both correlation estimation and reconstruction

Considering the importance of SI, several SI creation techniques have been presented in literatures such as: the motion compensated temporal interpolation (MCTI) [10], the motion compensated temporal filtering (MCTF) [11] and the SI fusion [2] Although these SI creation solutions are able to achieve SI with high quality, the complexity associated with each SI creation component is extremely large [4], thus, leading the difficulty in real-time video transmission

III PROPOSED COMPLEXITY CONTROLLED SI

CREATION SOLUTION This section describes the proposed complexity controlled SI creation solution Before that, the computational complexity associated with the SI creation processes is analyzed

A SI Complexity Analysis

Fig 3 shows the proposed SI creation architecture Here, we introduce a complexity controlling factor, gamma – γ, to adjust the SI computational complexity based on the energy allowed at each VSS node

Motion Estimation

MV Refinement

Motion Compensation

Motion Estimation

Motion Compensation

SI Fusion

B

X

f E

X

b E

X

SI

MCTI

MCTF

complexity control factor

 

1.ME 2.MVR 3.MC 4.SIF

Fig 3 Proposed SI creation solution (SigTelCom)

The input of the SI creation includes the BL

reconstructed frame, ˆX B and two forward, backward

EL decoded frames, ˆf, ˆb

creation includes four main stages: the motion

estimation (ME), the motion vector refinement

(MVR), the motion compensation (MC) and the SI

fusion (SIF)

To analyze the SI complexity, we measure the SI

processing time (PT SI), in seconds, for two video

surveillance sequences, Bank and Campus Fig 4

shows the percentage of complexity associated with

each SI component mentioned above

Fig 4 SI creation complexity analysis

From the obtained results, it can be concluded that

the ME stage usually consumes the highest

computational complexity percentage among other SI

creation components

B Proposed SI complexity controlled model

In ME process, the search range (SR) has been

confirmed as the main factor, which affects to both SI

quality and SI creation complexity [12] To study this

relationship, we measure the SI processing time for

several SR sizes, e.g., SR = {8; 12; 16; 20; 24; 28;

32} Two training video sequences are examined and

experimental results are shown in Fig 5

Fig 5 SI creation complexity with several SRs

From the results obtained in Fig 5, it is able to

conclude that the relationship between the SI

processing time and the SR size can be modeled as a

linear function:

SI

Here, {, } are two model parameters, which can

In this case, to control the computational complexity for the SI creation process, especially when the energy of VSS node is degraded along the time, a gamma factor, γ, with(0;1] is defined In this case, γ=1 corresponds to the maximum SR is used, i.e., SR = 32 Therefore, given a gamma factor, the optimal SR (SR opt) can be determined as:

(SR 32)

SI opt

PT



In summary, the proposed SI complexity control solution can be performed as the following procedure:

Procedure: SI complexity control

- Input: User preferred SI creation complexity, γ

(0;1]

- Output: SR optsize

1 Perform the SI creation with SR = 32

2 Compute PT SI(SR 32)

3 Read the complexity control factor, γ

4 Determine the optimal SR optas in (2)

IV PERFORMANCE EVALUATION This section evaluates the proposed complexity controlled SI creation solution Firstly, the test conditions are presented Secondly, the SI creation complexity with the proposed complexity controlled solution is evaluated Thirdly, the proposed method accuracy and the SI quality with different γ are shown Finally, the overall DSVC compression performance

is discussed

A Test conditions

As usual, four common surveillance video sequences [13] with different motion characteristics and contents are examined The detailed spatial, temporal resolutions, number of coded frames and other factors are provided in TABLE I while the first frame of each tested video is illustrated in Fig 6

Fig 6 Illustration of the first frame for the tested

surveillance videos

TABLE I Summary of Test Conditions

Spatial resolution,

temporal resolution,

number of frames

720×576, @30Hz,

201 frames

Quantization

Parameters

QPB = {38;34;30;26}

QPE = QPB – 4 Hardware

configuration  Processor: Intel® Core™

i7-4800MQ @2.7 GHz

 RAM: 8.00 GB

 System: Win 10, 64-bit

 Environment: Microsoft

Visual Studio 2017

Community

B SI creation complexity evaluation

To evaluate the SI creation complexity controlled

with the proposed method, the SI processing time is

measured for several gamma values Experimental

results are shown in Fig 7 confirm that the user

selected gamma factor is proportional to the SI

creation complexity For example, when the energy

of a VSS node reduces to 75% compared to the

beginning, it is able to adjust the SR size of SI

creation process using the proposed complexity

controlling procedure in section III.B In fact, this

complexity management is very important for VSS

applications, especially when the energy of each VSS

node is reduced along the time

Fig 7 SI creation complexity evaluation with

different γ

C SI quality evaluation

It should be noted that the complexity reduction

achieved with the small gamma factor is usually

associated with the quality degradation in the created

SI Fig 8 illustrates SI quality along frames with

several gamma values for four test sequences

The experimental results shown that the

complexity associated with the SI creation is usually

proportional to the quality of SI frame The higher

energy can be used for the SI creation process, the

better SI quality can be achieved

Fig 8 SI quality assessment with various gamma

values

D DSVC compression performance evaluation

Finally, it would not be completed without the compression performance evaluation for the proposed DSVC structure In this issue, we examine the DSVC compression performance for four test videos and four quantization parameters as described in TABLE I The common rate-distortion (RD) performance is compared between the proposed DSVC and the related benchmark, SHVC standard [6] Experimental results are shown in Fig 9

Fig 9 DSVC performance evaluation with different

SI creation From the obtained results, it can be concluded that the proposed DSVC solution importantly improves the compression performance than SHVC standard In addition, with the complexity control mechanism proposed in Section III.B, the proposed DSVC is even more flexible for many VSS applications

V CONCLUSIONS AND FUTURE WORKS Considering the need for a flexible DSVC structure in VSS application, we proposed in this paper a novel complexity controlled SI creation (SigTelCom)

solution The proposed complexity controlled

mechanism is performed based on the relationship

between the SI creation processing time and the size

of SR in ME stage The relationship between the SI

creation complexity and the SR is modeled as a linear

function A complexity controlling gamma factor is

defined to adjust the SI complexity and also the

overall DSVC processing time Experimental results

show the benefits of the proposed complexity

controlling solution, notably when the energy of VSS

node is attenuating along the time

REFERENCES

[1] M Valera and S Velastin, “Intelligent distributed

surveillance systems: A review,” IEE Proceedings -

Vision, Image and Signal Processing, vol 152, no 2, pp

192–204, Apr 2005

[2] X Hoang Van, J Ascenso, F Pereira, “HEVC backward

compatible scalability: A low encoding complexity

distributed video coding based approach,” Signal

Processing: Image Communication, vol 33 pp 51–70,

Apr 2015

[3] X HoangVan, J Ascenso, and F Pereira, “Optimal

Reconstruction for a HEVC Backward Compatible

Distributed Scalable Video Codec,” IEEE Visual

Communication and Image Processing (VCIP), Valletta,

Malta, Dec 2014

[4] X HoangVan, J Ascenso, F Pereira, “Adaptive Scalable

Video Coding: a HEVC based Framework Combining the

Predictive and Distributed Paradigms”, IEEE

Transactions on Circuits and Systems for Video

Technology, vol 27, no 8, pp 1761-1776, Aug 2017

[5] L Dao Thi Hue, et al “Efficient and low complexity

surveillance video compression using distributed scalable

video coding” VNU Journal of Science: Comp Science &

Com Eng., Vol 2018

[6] G J Sullivan, J.-R Ohm, W.-J Han, and T Wiegand,

“Overview of the High Efficiency Video Coding (HEVC)

Standard,” IEEE Transactions on Circuits and Systems for

Video Technology, vol 22, no 12, pp 1649-1668, Dec

2012

[7] J M Boyce, Y Ye, J Chen, and A K Ramasubramonian, “Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding

Standard,” IEEE Transactions on Circuits and Systems for

Video Technology, Vol 26, Issue 1, pp 20-34, Sept 2015

[8] F Pereira, C Brites, J Ascenso, M Tagliasacchi

“Wyner–Ziv video coding: a review of the early architectures and further developments,” IEEE International Conference on Multimedia and Expo (ICME), Hannover, Germany, June 2008

[9] A D Wyner, J Ziv, “The rate-distortion function for

source coding with side information at the decoder,” IEEE

Information Theory Society, vol 22, no 1, pp 1–10, 1976

[10] J Ascenso, C Brites, and F Pereira, “Improving frame interpolation with spatial motion smoothing for pixel

domain distributed video coding,” 5th EURASIP

Conference on Speech and Image Processing, Multimedia Communications and Services, Slovak, Jul 2005

[11] J Ascenso, C Brites, and F Pereira, “A flexible side information generation framework for distributed video

coding,” Multimedia Tools and Applications, vol 48, no

9, pp 381-409, 2009

[12] X HoangVan, H Phi Cong, “A novel content adaptive search strategy for low complexity frame rate up

conversion,” 2nd International Conference on Recent

Advances in Signal Processing, Telecommunications & Computing (SigTelCom), Ho Chi Minh City, Vietnam,

Jan 2018

http://mlg.idm.pku.edu.cn/-resources/pku-svd-a.html