Contrast enhancement framework for suppressing JPEG artifacts 1

is to solve this problem starting by examining the current compression artifacts reduction algorithms on contrast enhanced images.. To deal with this problem, we have proposed the image

CONTRAST ENHANCEMENT FRAMEWORK FOR

SUPPRESSING JPEG ARTIFACTS

GUO FANGFANG (B.Sc., ShanDong University of China, 2012)

A THESIS SUBMITTED FOR THE DEGREE OF MASTER OF SCIENCE

DEPARTMENT OF COMPUTER SCIENCE

NATIONAL UNIVERSITY OF SINGAPORE

2014

@ 2014, GUO Fangfang

Declaration

Thereby declare that this thesis is my original work and it has

been written by me in its entirety I have duly acknowledged all

the sources of information which have been used in the thesis

This thesis has also not been submitted for any degree in

any university previously

To my parents.

Acknowledgements

First, my sincere gratitude to my advisor Dr Michael S Brown for his patient guidance during my M.Sc candidature He enlightens me in the right direction I am thankful for all of his encouraging advice, shared experiences and technical support and I definitely benefited a lot

To complete this thesis, my colleague in the Computer Vision Lab, Li

Yu, has helped me a lot I truly appreciate his valuable suggestions, pa- tient explanations, and contributions to my research work It is always

my pleasure to collaborate with these brilliant people and to work on existing research topics together

I also feel thankful for all my colleagues in the lab, with whom I have spent two years happily during my M.Sc candidature

Finally, I would like to thank my parents and my friends for their

continuous support.

Contents

List of lables Ặ.Ặ Q Q Q HQ HQ HQ Hi List of Flgures Ặ Q Q HQ HQ HH HH HQ kia Introduction

1.1 Motivation .0 0.000 eee ee et es 1.2 Research Problem Statement 00000 eee eeeeees

Background and Preliminaries

2.1 RelatedConcepts HQ HQ HQ HH HH 2.2 Image ContrastEnhancement co 2.2.1 GlobalContrastEnhancement 2.2.2 LocalContrastEnhancement

2.3 JPEG ArtifactsReducton Ặ c Q Q ee

2.3.1 Deblockng Q HQ HH HH ko 2.3.2 Derinping ch HQ HH HH HQ kg kg

"Vk n ee TH Proposed Method

3.2 Structure-lexture Decomposiion co 3.3 Reducing Artifacts in the lextureLayer 3.3.1 Scene DetailExtracion Ặ Ặ So 3.3.2 Block Artifacts Reduction 000 ee uae 3.4 Layer Recomposition 6.6 ee ee

11

CONTENTS

4 Evaluation

4.1 Experiment Design

4.2 Experimental Results

4.2.1 Tonecurveadjustment -

4.2.2 Dehazing and Underwater Image Enhancement

4.3 Discussion and Analysis 5 Challenges and Future Work 5.1 Challenges

5.2 FutureWork

5.2.1 Complete Previous Work .-

5.2.2 Extension to Generic Compression Scheme

5.2.3 Extension to Video Processing

6 Conclusions 6.1 Summary

6.2 Contributions

6.3 Discussion and Limitations

Bibliography

iv

27

27

28

30

33

37

43

43

44

45

45

46

49

49

51

51 53

is to solve this problem starting by examining the current compression artifacts reduction algorithms on contrast enhanced images The initial results show that these current algorithms need to be improved in this case The problem here is that the imperceivable compression artifacts

in low-contrast images would be unintentionally boosted when we try

to only enhance the images appearance The challenge in this problem

is that the two tasks we want to achieve are functionally opposite

On one hand, we aim to enhance the contrast of the image content

as much as possible On the other hand, within the same image, we

want to suppress the contrast of the compression artifacts If these two tasks are processed sequentially, as pre- or post-processing, the results are not likely to be optimum The process of artifact removal as pre- processing will remove the image content that have low contrast, and as post-processing will be affected by the enhanced artifacts The essential problem in these kinds of methods is that they process the image content

and noise all together no matter in the contrast enhancement step or in the artifacts reduction step

To deal with this problem, we have proposed the image decomposition based framework to supress artifacts appearing in JPEG images that becomes prominently visible when contrast enhancement is applied to the images While the proposed framework is admittedly engineering in nature, our strategy of using structure and texture layer decomposition enables us to process them independently to each other, and to reduce the compression artifacts in parallel with contrast enhancement

Experiments show that our integrated framework can produce com- pelling results compared with generic deblocking algorithms applied sequentially with a contrast enhancement procedure

vi

List of Tables

4.1 Average runtime comparison

4.2 Quantitative comparison

Vii

LIST OF TABLES

vill

List of Figures

1.1

3.1

3.2

3.3

3.4

4.1

4.2

4.3

4.4

4.5

4.6

4.7

4.8

4.9

Traditional contrast enhancement methods on the low-quality JPEG

Pipeline of the proposed method

Ilustration of image decomposition .-

The scene details map generatlon

The effect of deblocking on the texture layer

The comparison of existing algorithms and our method on tone- curve adjustment (case1) 2 ee ee ee The comparison of existing algorithms and our method on tone- curve adjustment (case2) .Ặ Ặ eee The results of our methods on different compression levels images More results of ourmethod

Comparison of existing dehazing methods

More comparisons of existing dehazing methods Comparison of existing algorithms and our method on dehazing (Casel) ee ee ee eee Comparison of existing algorithms and our method on dehazing

An illustration of our framework applied to underwater image en- hancement 0.0 00 eee ee ee ee te te ke ht we ne

ix

LIST OF FIGURES

Chapter 1

Introduction

1.1 Motivation

Contrast enhancement is frequently referred to as one of the most important issues

in low-level computer vision The purpose of image enhancement is to improve the interpretation or perception of information contained in the image for human viewers, or to provide a better input for other automated image processing sys- tems, such as color segmentation, edge detection, image sharpening, etc Contrast enhancement can be applied either explicitly or implicitly We can boost an image’s global contrast explicitly through histogram equalization, tone-curve adjustment

or gradient-based enhancement Besides, improving the visibility of images de-

graded by environment such as haze, fog, rain, and underwater is in nature an

spatially varying implicitly contrast enhancement

However, all the contrast enhancement algorithms are based on the assumption that the input image is not highly compressed and free of significant noise If the input image is highly compressed, when the algorithm enhances the contrast of

CHAPTER 1 Introduction

the content in the image, it unintentionally boosts the unsightly image artifacts, especially artifacts due to JPEG compression In reality, in order to improve the speed of image stream transmission on Internet, compression schemes are usually applied to high-quality images to make the file size of images smaller Moreover, the images and videos from surveillance camera are often compressed significantly When we want to enhance the contrast of these low quality images, the results of current state-of-the-art algorithms are far from visually pleasing due to the obvious boosted artifacts, which can be seen in Figure 1.1

There are many well-known algorithms dealing with JPEG artifacts, such as the re-application of JPEG [26], Field of Experts [35], Shape-Adaptive DCT deblocking and denoising [10] and learning-based image denoising [2] The intuitive idea is to apply JPEG artifacts reduction algorithms before or after the contrast enhancement operation However, after we have tried applying the denoising methods as pre-

or post- processing, they do not produce visually pleasant results When applied before the enhancement process, the algorithms may over-smooth the texture of the image When applied as a post-processing procedure, the algorithms could not effectively remove the boosted artifacts

To solve this problem, we propose a layer decomposition based framework which integrates contrast enhancement and JPEG artifacts reduction together It can enhance the contrast of the low-quality images and meanwhile suppress the JPEG compression artifacts This framework is suitable for either global contrast enhancement or local contrast enhancement.

1.2 Research Problem Statement

1.2 Research Problem Statement

Our research goal is to deal with the boosted JPEG artifacts during the contrast enhancement procedure Given a low-quality image, how can we produce the

enhanced result free of noticeable JPEG artifacts? On one hand, we want to enhance

the contrast of the content in the image as much as possible On the other hand, to void the block and ringing artifacts being boosted, we have to suppress the artifacts

to the maximum extent Based on this objective, we state several research problems

in this thesis

Firstly, we need to separate the JPEG artifacts from the image content properly Once we separate these two layers, we can do different operations on each layer in-

CHAPTER 1 Introduction

dividually In our case, we can perform contrast enhancement on the image content

to get the expected enhanced results, and perform the reduction of JPEG artifacts

on the texture layer to suppress artifacts Recomposition of the two processed layers can achieve the aforementioned objective

Secondly, it is very important to combine the two layers properly together

to get the final result The naive solution that directly adds the two processed layers together would introduce strong ringing artifacts Another problem is that the block artifacts in the homogeneous region and the image details are difficult

to differentiate in the same image If we deblock too much, the block artifacts

will be removed as expected, however, the details are over smoothed at the same time However, if we deblock insufficiently, the block artifacts will not be removed

effectively To prevent such degradation problems, we need to compute a mask which separates the main objects from the homogeneous regions that should be

omitted when added back to the final result

1.3 Overview

The rest of this thesis is organized as follows Chapter 2 presents a literature review related to our research The detailed technologies we have adopted in our proposed methods are illustrated in Chapter 3 In Chapter 4, we show the experiments conducted as well as the evaluations of our method In addition, the challenges and future works are discussed in Chapter 5 Finally, in the last chapter,

we draw the conclusions and discuss the limitations of this thesis.

Chapter 2

Background and Preliminaries

2.1 Related Concepts

e Image Contrast Enhancement

Image contrast enhancement is a process involving changing the pixels’ inten- sity of the input images, so that the output image looks subjectively better [12] This procedure can be performed either explicitly or implicitly The explicit common methods used for improving contrast in digital images are histogram equalization (HE) and tone curves These kind of methods are spatially in- variant contrast enhancements Some examples of the latter are dehazing and underwater image enhancement Due to the substantial presence of particles

in the atmosphere that absorb and scatter light, the degradation of the con- trast of the haze-free image mainly depends on the distance of the objects to

the camera, which tends to be smooth This kind of contrast enhancement

is spatially variant and can be considered as local contrast enhancement These operations are commonly employed for image enhancement because

CHAPTER 2 Background and Preliminaries

of their simplicity and great performance on high-quality images that are free

of obvious JPEG artifacts

e JPEG Compression Scheme

A brief overview of lossy image compression and the JPEG standard is intro- duced here The purpose of image compression is to represent images with

as less data as possible in order to save storage costs or transmission time Lossy image compression is to remove the high-frequency (noise-like) details that the human eye typically does not notice The lossy compression meth- ods commonly used are Fourier-related transform coding such as discrete

cosine transform (DCT, used in JPEG, MPEG-1, MPEG-2, H.261, H.263 and

its descendants) and wavelet transform (used in JPEG 2000) The degree of compression can be adjusted, allowing a selectable trade-off between storage size and image quality The detailed discussion of the theory behind quan- tization and justification of the usage of linear transforms can be found in

[6]

The JPEG encoder partitions the image into 8x8 blocks of pixels To each block,

it first applies a 2-dimensional DCT individually, followed by the quantization

of the DCT coefficients element-wise by a 8x8 quantization matrix The high-frequency image content can be quantized more coarsely than the low- frequency content since there is a smaller amount of energy packed in the high-frequency bands of most natural images The human visual system is also less sensitive to quantization loss in the high-frequency bands

The quantized DCT coefficients form a matrix that is usually sparse, i.e there are many Zeros in it, especially in the high frequency bands The elements of

6