A Novel Technique in Error Propagation Resilience for H.264 Video Coding Based on FMO and Intra Refresh Rate Ky thuat giam 16i Ian truyin cho ma hoa video theo chuSn H.264 diia tren ky
Trang 1A Novel Technique in Error Propagation Resilience for H.264 Video
Coding Based on FMO and Intra Refresh Rate
Ky thuat giam 16i Ian truyin cho ma hoa video theo chuSn H.264 diia tren ky thu^t FMO
va ky thuat ma hoa Intra
Dang Hoai Bac^*, Vu Huu Tien^
' Electronics Engineering Department, Posts and Telecommunications Insntute ofTechnology, Hanoi Viet Nam
^ Multimedia Department, Posts and Telecommunications Institute ofTechnology Hanoi, Viet Nam
Received: April 22, 2014; accepted- August 25 2014
Abstract
In video coding, due to the references to neighboor frames while coding the current frame, if there is an error occuring in a neighboor frame, the current frame will be affected This error propagation phenomenon
is one of issues in video coding needs to be resolved In this work, we propose an error resilience scheme for wireless video coding based on adaptive Frexible Macroblock Ordering (FMO) and intra refresh FMO explicit map is generated frame-by-frame using the estimated prior information The prior information involves the estimation of locations of guards and bursts of errors in the channel using three-state markov model and the estimation of the error propagation effects (EEP) from the current to the next frame Intra refresh is also used to reduce the dependency between frames therefore could stop error propagation ami
a suitable intra refresh rate is selected based on channel state to reduce the effect of error propagation compared to some other methods that have not taken channel condition and error propagation into consideration in generating FMO map
Keywords: H,264, FMO, Intra Refresh
Tom tat
Trong qua trinh ma hoa tin hieu video, khi ma hoa mot khung thi khung hinh lan can dcrac tham chiiu S udc luiyng chuyin dong va dir doan khung hinh hien tai Vi vay trong tnrdng hap khung hinh tru^ do bi loi thi khung hinh hien tai cung se bi anh hudng Hien tuang nay dirac goi la lan truyin loi giira cac khung nhim giam anh hiK/ng cua idi lan truyin khi truyin tin hi§u video qua mang vo tuyin Thuat toan sifdiing phuong phap tao sa dd FMO cho tCrng frame dua tren viec du doan trang thai kenh truyin Di tao sa do FMO, muc do anh hudng cua Idi truyin lan duac su' diing di danh gia mire dd quan trt^ng cua tCrng /c/ioi anh trong khung hinh hi^n tai Dua v^o md hinh Markov ba trang thai, tr^ng thai kenh truyin duv^c du doan Cac khoi anh quan trong cua khung hinh se dwac truyin di khi kenh truyin d trang thai tot C^c kti6i anh it quan trong han dwac truyin di khi kenh truyin a trang thai bi anh htrang nhiiu bai fading Ngoai ra, mot s6 khdi anh quan trong nhit trong khung hinh se dtrac ma hda a chi do Intra coding Kit qua rr\o phong da chi ra ring thuat toan dtrac di xuit cho hinh anh chat Iwang video sau giai m§ tdt han mot so thuat toan trwdc dd khi khdng xet din tinh trang idi truyin lan va tinh trang cua kenh truyin
Ju khoa: H 264, FMO, Intra Refresh
1 Introduction number of slice groups is limited to 8 for each picture
FMO [1], one ofnew techniques in H.264/AVC ' ° """""^ " " P ' ' " ' ' " o ' : " " " " schemes B,
standard [2], is proposed with the aitn at reducing ^ " ^ " " E » s " ' ^ " ' ^ MBAmap, the error MBs m burst errors while transmitting video signal over error fr"™ » « dispersed Consequently, the end prone channel, FMO consists of deciding on which
shce each macroblock of the image belongs Each
concealment algonthm in decoder can recover the lost information from neighboring MB that is
macroblock can be assigned freely to a slice group -^
using an MBAmap (MacroBlock Allocation map) In the previous approaches, to design slice group Tbe MBAmap consists of an identification number maps, an indicator is used to define the importance of for each macroblock of tbe image that specifies on an MB The indicators may be MB impact factor [1] which slice group that macroblock belongs The and bh count [3], In these methods, a simple way is
used to generate FMO map in which MBs with h i ^ importance are arranged consequently to 8 slice
* Corresponding author Tel: (+84) 903.284.728 groups However, it's not guaranteed that important
Trang 2MBs are protected from error Due to variable length
MBs IS error, tbe next MBs are also affected
Therefore these methods can only reduce number of
loss important MBs
In video coding, error propagation occurs due to
the relationship between frames in a slice group To
stop error propagation effectively, intra refresh
algorithm is used However, the selecting a suitable
intra refresh rate is a problem needed to be
considered Intra MBs can effectively stop error
propagation, but the number of intra MBs in a frame
is limited by compression efficiency Coding
efficiency will be reduced if the mfra refresh rate is
high because of rate-distortion optimization
Moreover, with limited target bit rate allocated for
each frame, a frame with high number of intra MBs
consumes high target bit and thus affects to the target
benefit of reducing error propagation effect and the
drawback of using a high number of intra coded MBs
in a frame
There exist some researches focusing on
resolving intra refreshment in the previous studies In
[4], a fix number of MBs with the highest distortion
Another study in [5] chooses the suitable MBs for
coding mode However, the algorithm has to try all
possible mtra and inter modes and thus may cause
delay in real-time appheations
In this work, the method of FMO map
generation in [8] is referred However, the method for
MB more exactly The final results of proposed
method are compared with the results in [8] to show
the evidence of improvement
The remainder of the paper is organized as
follows Three-state Markov model and proposed
sections are introduced in Section II Section III
presents the proposed method to generate FMO
results and discussions Finally, conclusions are given
on Section V
2 Error Burst Location Estimation
2.1 Three-state Markov Model
In this work, transmission channel state is
predicted by using three-state Markov model [6-7],
Based on chaimle state, FMO explicit map is
generated with targets on reducing effects of
transmission error for important MBs,
In the proposed method, instead of using model
in bit level, three-state Markov model in packet level
is used Figure I shows an example of the packet sequence for an error chaimel We define the following defmitions at packet level, similar to [6], and a burst section is defined as the section sandwiched between guard sections The minimum guard length is tbe minimum length of a guard section In this work, the minimum guard length of 10 packets is used as a result from empirical studies consecutive error-free packets The run length is defined as a length from an error packet to the next error packet excluding the first error packet The first
return probability P(t) is defined as the occurrence
first return probability is obtained as the cumulation
of the first return probability
|< ft 1
i|oooooi|i|aoi
Burst lens tti 0' Correcl packet
1 Error packet
' 0 0 0 0 0 0 - 0 5 0 0
Fig 1 Packet sequence for an error charmel
Fig.2 Three-state Markov Model Figure 2 shows the transition probabilities of a
three-state Markov model where C; and Cz show error-free states which are state 1 and state 2, and E shows the error state which is state 3 Ci shows error-free state in a guard section while C2 and E show error-free and error state in a burst section p„„i is probability of transition from state n to state m p„,„''^
is the probability of the case in which there are (i-2}
first return probabilities are computed as shown in Eq.(I),
n i ) =p<.E/E) = p,,
P(T) = p(C,,E/E) + p(C,,E/ E)
^ P31P13 + P32
/'CO ^ M C , , , , , £ " / £ ) + p{C;, ,C^,E/E-)
(1)
, where p{a 10) means that ^ is the first state and then
the sequence of a occurs The fransition probabilities
Trang 3f,(i-'i}Pi<)
= i-Pi,-Pii
(2)
P22 = 1 - P ; 3
Pu = l " / ' l 3
, where Lmm is the mimmum guard length According
to [6], the average guard length, Lc, and average burst
length LB, are computed as
2.2, Error Burst locating
Assume that before encoding the current frame,
encoder receives the feedback information including
number and position of error packets in the previous
the La and LB of channel These values are used for
the whole transmission duration of current frame and
then are updated when encoder receives the next
feedback information To estimate burst section or
guard section in the current frame, the position of last
burst or guard section of the previous frame are
located If the border of the previous frame and the
current frame is in a guard section, the distance from
the last burst in the previous frame to the first burst in
the current frame is LG packets {see Fig, 3)
Otherwise, the distance from the last guard m the
previous frame to the first guard in the current fi^me
is LB packets The next sections in the current frame
are located from position of the first section Tbe length of current frame is esrimated by bitcount in the first pass of encoder
Fig 4 and 5 illusfrate the estimation of error burst for 100 frames of Akiyo sequence m fest and slow fading Because encoder needs feedbadi information from previous frames therefore estimation is started at the fourth frame of video sequences The results show that in slow fading, because of the lower variable in length of burst and guard section, the estimation is more precise than thai
m fast fading case
3 Explicit F M O M a p Generation
In this section, the in^ortance of an MB is estimated based on the distorrion and effect of error propagation within a frame and between the current into slide group according to the rule: The imporlanl MBs are inserted in the predicted error-free period of channel and the less important MBs are inserted ID the predicted error penods (Fig, 6) In general, encoder needs more time to create explicit FMO than information of channel state is predicted Thus, explicit FMO map can be created before coding the current frame Consequently, codec performance is not affected much by the explicit FMO map generation time
Fig 3 Estimating AGL and ABL for the current frame
Estrmated burst section locaton
" l l
^'^
"ll
n n n 1 , n i
Actual bufst section locatxms
1 , n ,
" " " ^ n
n
n,
|- Esbmated b u r s t s e d i o n l o c B b o n
Fnnnmmiii iraraimonmi >im l a n 2UJ a
Packs)
r- A c t u a l burst secSon kicabons
FiiniiJiiiiii 1 III III 0.11111111
Packet
r E m o r i n a s t i m d i o n
M 1 U III I III 1 III II,
Fig 4 Estimation of burst and guard sections for Akiyo Fig 5 Estimation of burst and guard sections for Akiyi
Trang 43.1 Distortion and E r r o r Propagation Analysis
Fig.6 Arranging MBs into slice groups in the current frame
n bits
Fig 7 Bit number of first k MBs in a slice group
Best match
macroblock ofj MB
Current frame
(k)
Next frame Fig 8 Effect of a loss MB to an MB in the next frame
In this section, method for estimating distortion
and error propagation error is based on the method in
[9] However, instead of using a fixed number of
MBs m each slice group for all frames as [9], in this
work, the number of MBs for each slice group is
varied according to the channel state
Assuming single bit error occurs independently
in the bit sfream and error probability of single bit
is p , we will estimate the probability that kth MB in
be error in the event that at least a single bit error
occurred in first k MBs of slice group Assume that
the first k MBs in slice groups have n bits as shown
in Fig 7 Because length of the current slice group
depends on the channel state, it's assummed that the
number of MBs in the current frame equals to the
number of MBs in the co-located slice groups of the
previous frame
in (4):
The error probability of kth MB is computed a
P(MB^) = Y^C„p'q = 1 - 9 " (4)
In the current frame, it is assumed that MB ith
has error To stop this error that may be propagated to
the next frame, non motion-compensated error
concealment is used by replacing the MB ith with the
co-located MB in the previous frame The
distortion-from-error-concealment is computed in (5),
^ £ C , = J^\f,(x,y)-f,_,{x,y)\ (5)
where frame kth and (k-l)lh are the current frame and previous frame, respectively, f(x,y) is reconstructed pixel value at the coordinate (x.y) and
M is damaged MB ith
To estimate the effect of error propagation, the
distortion-from-error-propagation, D^p, is defined
i>£P IS tbe distortion caused by MB ith in the current frame to MB Jth in the next frame through
motion compensation Assuming that the best match
macroblock of MB jth in the next frame covers MB
ith in the current frame and this reference area is
denoted by Las shown in Fig 8
Let Rj^(I, j) denotes the distortion of MB j'lk caused by L in case MB ith has no error fl^C/,/) is
computed as shown in (6),
(x.ysLj
, where (x.y) is the pixel coordinate in the current
frame and ( M I ^ , M ) ^ ) is the motion vector of MB
jth
Let Rkii,j) denotes the distortion of MB jth caused by L in case MB ith has error R],(i,j) is
computed in (7)
K(hJ)= J}fk^iix,y)-f,^^(x-MV,_^~MV^,)\ (7)
The distortion-from-error-propagation caused by
MB ilh in kth frame to (k+l)th frame is computed as
shown in (8),
, where W is a set of MBs m the next frame which
reference MB ith in the current frame
Finally, to estimate the importance of an MB in the current frame, we have to consider the total distortion caused by this MB in case it has ertor The
total distortion of error MB ilh is computed as shown
in (9)
Based on the total distortion, MBs are sorted in
Trang 5consecutively to have an initial slice group map
Seeing that losing one MB will affect the rest of MBs
in the slice group, we now compute the overall
distortion of a MB, D^^^^n, from the initial slice
group map:
^o> «»=i](i^rM./,-^('WB,)) (10)
Where i denotes the set of MByVA to the last MB Nth
in a slice group, D^g^^i is the total distortion of MB
computed from (9) and P{MB,)is error probability
MB ith in the slice group computed in (4)
In this work, two-pass architecture is used for
encoding In the first pass, all MBs in P frame are
encoded m inter mode After that, bitcount and
overall distortion estimation of each MB in a frame
are computed In the second pass, after getting
feedback information from the previous frame, Lg
and Lg are computed and then positions of burst and
guard sections in the current frame are located Based
on overall distortion compured in (10), MBs are
selected to fill in each section until the total bitcount
of selected MBs equals to length of section The
detail of algorithm is described as follow:
Step I:
Sorting MBs of the current frame in descend
order ofDoverati to generate a list of MBs
Estimating LG and LB of the channel
Step2:
If the current section is not the last section {
If the section is burst {
Arranging MBs of the list into the current section
until the total number of packet of MB is larger
than LB
}
Elsef
Arranging MBs of the list into the current section
until the total number of packet of MB is larger
than
LG-}
)
Else{
Go to the Step 1 for the next frame
I
3.2 Adaptive Explicit FMO Map
Similar to method in [8], the number of slice
groups in a section , Nsis, is computed as shown in eq
, where be means the largest integer no greater than
X H is the number of MBs selected to fill in the
current section In guard section, some MBs with highest overall distortion are selected for intra coding mode The number of infra MBs in guard secHon,
^Intra' ^^ computcd as shown in eq (12)
The lager value of a is, the less effect of error
propagation is However, if value o f a is increased, the bit rate is increased and compression efficiency is reduced consequently To balance the compression efficiency and effect of error propagation, a is refresh rate depends on the number of guard sections from 0 (if there is no guard section in transmission duration of frame) to 16 (if the whole frame is because intra coded MBs are dispersed in some slice more guaranteed than the case without knowing aboul guard and burst of error
4 Simulation Results and Discussion
4.1 Experiment Setup
In this experiment, the reference software JM 9.2 is used At the decoder, the non-motion following video sequences are used in experiment' Akiyo, Foreman, Carphone and Claire Each sequence was encoded for a total of 100 frames at a frame rate of 10 frames per second Rate control was enabled and bit rate was set at 32kbps The default encoder parameters were used with the exception FMO related parameters; see [10] for detaileil information about the H.264 JM encoder parameters channel, the Rayleigh fading wireless channd simulator was used in this simulation To simulate fit effects of slow and fast fading channels, 1^ maximum Doppler frequency parameter was set tc [ I I ] ,
4.1 Results and Analysis
In this work, the results are compared to mefta
of [8] m which the error propagation within a sliis the total number of undecodable MBs are used as ft performance metrics m quantifying the effectiven|
Trang 6Table 1 Comparison of average PSNR in case of Table 2 Comparison of average PSNR in case ol using FMO without Intra refresh using FMO without Infra refresh
PSNR
No FMO
Method in [8]
Proposed
method
Akiyo
Slow
30.22
35.02
35.02
Fast
26.59
32.71
32,71
Foreman Slow
17 67 22.9
Fast 15.54 19.5
19.8
PSNR
No FMO
Method in 181
Proposed
method
Akiyo
Slow
30.22
34.58
34.61
Fast
26.59
33.13
33.56
Foreman Slow 17.67 24.83 25,3
Fast 15.54 19.9
20,10
PSNR
No FMO Method in [8]
Proposed method
Claire Slow 30.22 33.9
34,01
Fast 26.59 30.45
32.01
Foreman Slow 17.67 27.58
27,94
Fast 15.54 24.61
25,14
Table 3 Comparison of average PSNR in case of
using both FMO and Intra refresh
Table 4 Comparison of average PSNR in case of using both FMO and Intra refresh
PSNR
No FMO Method in [8]
Proposed method
Carphone Slow 28.37 33.08
33,88
Fast 24.65 31.63
31,81
Claire Slow 22.41 28.39 28.02
Fast 19.75 26.15 27,05
I Fig 9 PSNR comparison of Carphone in slow fading Fig 10 PSNR comparison of Carphone in fast fading
Table 1 & 2 show that in the case of without
I Infra refresh, PSNR value of the proposed method is
higher than that of the previous method The reason
; of this improvement is that the algorithm for MB
I evaluation of the proposed method is more precise In
I case of using Infra refresh without FMO, the results
1 of the proposed method are the same as the method in
! [8] because the value of a in eq, 12 is the constant
Table 3 & 4 show the average PSNR of video in
the scenario of slow and fast fading channels The
simulation results show that for both cases of fading,
PSNR is improved in the proposed method Although
the same slice group map is used, the proposed
method has selected suitable MBs for intra coding
slice group Therefore, the estimation of important
MBs in the proposed method is more precise
'proposed method is low because the quality of video
'ithe location of bit errors and the ertor concealment
'method applied
Figure 9 and 10 shows the average PSNR curve
of Carphone test sequence in the slow and fast fading
improvement of thc proposed method can be clearly accurate method in estimating the importance of MB
to generate slice group map and in adjusting the number of infra coded MB for each frame Additionally, by using FMO, the important MBs are dispersed in the whole frame, thus reducmg the loss
of important MBs Consequentiy, the PSNR of proposed method is increased,
5 Conclusions
In this work, besides the method of arranging MBs according to channel state as proposed in [8], and between frames are imporved to measure the intra refresh rate is also used to reduce the effect ol error propagation Experimental results show that om proposed method gains some improvements in terms
of PSNR as compared [8] that have not taken erroi propagation between MBs in slice group intc consideration in generating FMO map
References
[1] P Lambert, W.D Neve, Y Dhondt, and R \ dt
Trang 7H 264/AVC", J, of Visual Comm And Image
Representation, vol 17, pp 358-375, Apnl 2006
[2] ITU-T Recommendation H.246, Advance video
coding for generic audiovisual}, March 2005
[3] W Hantanong, and S Aramvith, "Analysis of
Macroblock-to-slice Group Mapping for H.264 Video
Transmission over Packet-Based Wireless Fading
Channel", 48th Midwest Sympsium on Ctkt and Sys,,
vol,2,pp 1541-1544, August 2005
[4] B Girod and N Farber, "Error-Resilient
standard-compliant video coding" Available at link:
http,//citeseerx.istpsu,edu/viewdoc/summary?doi=10
1.I.51.684I
[5] J Xu, Z Wu, "Joint Adaptive Intra Refreshment and
Unequally Error Protection Algonihms for Robust
Transmission of H,264/AVC Video", Multimedia and
Expo, 2006 IEEE International Conference, July
2006, pp, 693-696
[6] T, Sato K Tokuda, M Kawabe, and T Kato,
"Simulation of burst error models and adaptive error
analog cellular system", IEEE Trans Veh Teehnol.,
vol VT-40, pp 443-^52, May 1991
[7] T Sato, M Kawabe, T Kalo and A Fukasawa,
"Throughput Analysis Method for Hybrid ARQ
Schemes Over Burst Error Channles", IEEE Trans
Veh Teehnol., vol 40, no l,pp, 110-118, Feb 1993
[8] V.H Tien, S Aramvith and Y.Miyanaga, "New Error
Resilience Technique Using Adaptive FMO and Intra
Refresh for H264 Video Transmission", lEICE
Trans on Fundamentals of Electronics,
Communications and Computer Sciences, Vol.E94-A,
No.8,pp,1647-l655
[9] V,H, Tien and S Aramvith, "An error resilience
technique based on FMO and error propagation for
H.264 video coding in error-prone channels",
Multunedia and Expo, ICME 2009, pp 205 - 208,
June 2009
[10] A Michael, K, Shnng, and G Sullivan, "Proposed
H.264/MPEG-4 AVC Reference Software Manual",
Jomt Video Team, Doc JVTM012, Palma (Spain),
Oct,2004
[11] T C, Chen, et al,, "A Real-Time Software Based
End-To-End Wireless Visual Communications
Simulation Platform", Proc SPIE Visual Comms and
Image Proc, vol.3, pp 1068-1074, May 1995