Luận văn content based video indexing and retrieval

Thesis for Degree of duster - Content-based videu indexing oud retrrevial Figure I1T-14 Video frame from video sequence | Figure I-15 Video frame from video scquonec 2 Figure II-16 Key f

BỘ GIÁO DỤC VÀ ĐÀO TẠO

TRUONG DAI HOC BACH KHOA HA NOI

PHAM QUANG HAI

CONTENT-BASED VIDEO INDEXING AND RETRIEVAL

LUẬN VĂN THẠC SĨ KỸ THUẬT

xU LY THONG TIN Va TRUYEN THONG

Hà Nội - 2005

BỘ GIÁO DỤC VÀ ĐÀO TẠO

TRUONG DAI HOC BACH KHOA HA NOI

PHAM QUANG HAI

CONTENT-BASED VIDEO INDEXING AND RETRIEVAL

LUAN VAN THAC SI KY THUAT

XU! LY THONG TIN Va TRUYEN THONG

NGUOI HUONG DAN KHOA HOC:

Alain Boucher

Hà Nội - 2005

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Abstract

Video indexmg and retrieval is an important clement of any large multimedia database Because video contains huge information and has large store requirement, the research on video is still continuous and

opens ‘his thesis works at low-level features of video, concentrates to

the corner feature of frame Comparison of corners between frames brings back to us the ability of detection for cut, gradual transitions and even rich type of wipes of shot transition in video Continue to use comer-based motion combining with histogram features, by measuring

distance of how far the motion move, key frames is selected, and it is

ready for indexing and retrieval application

Other side of work is using segmentation of each frame and

merging all regions which have the same motion By this way, | would like to separate regions in frame into layers and it will used to indexing key objects

One chapter of this thesis is reserved for learning how to index and retrieve in video It is an overview of video indexing and retrieval system: what they did, what they are doing and how they will do This thesis is expected to contribute usefully to multimedia system at MICA

List of abbreviations v List of figures - - - cet Lisl of tables

Chapter 1 Introduction

L1 Content-based Video Indexing and Retrieval (CB VIR)

12 Aims and Objectives

112 Content-based video indexing and retrieval system

IL2.1 Video sequance structure

112.2, Video data classification,

112.3 Camera operations -

11.2.4, Introduction lo CRVIR systom

112.5 CBVIR Archilecture

3 Features Extraction

I4 Structure Analysi:

TIA1 Shot Transitions classification

1142 Shot Netzetion Teelmiques

1.6.1 Introduction to Video Abstraction

1162 Key frame extraction: -

1163 Video Abstraction - - -

TL7 Vidco Iudexing, Refricval, and Browsing - a AB U8 Thesis Scope 44 Chapter IIL Video Indexing by Camera motions using Comer-based motion vector

TL2 Video and Image Parsing in MICA

HIL2.1 MPEG2 Video Decoder

11.2.3 Video and Image Processing akrry Combination so AB

HL3.1 Hamis Corner points deteotor co 47

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

11.3.2 Corespond poituls matching

TI4 — Shol Transitions dotzction

11.4.1 Shot cut Detection algorithm

11.42 Shot cut detection description

143 Results and evaluation

TIS Video Indexing

1.5.1 Motion Characterization

11.5.2 Comer-based motion vector

1153 Global Motion calculation

W154 Key frame extraction

1.5.5 Problem in object extraction

iv

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Figure I1T-14 Video frame from video sequence |

Figure I-15 Video frame from video scquonec 2

Figure II-16 Key ftarne selection ñom video maosaic

Figure 1LL-17 Key frames is selected from motion graph

igure 1-18 Complicated motion graph from video

Figure ITT-19 cases of vector graph -

Figure I-20 Results of key frame selection

Figure I-21 Hierarchical indexing for CBVIR system

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Acknowledgements

This work is a part in Multimedia Information Communication

Application (MICA) research central

First of all, I would like to thank to Dr Alain Boucher, IT lecturer

at Institut de la Francophonie pour I’ Informatique (JF1), Vietnam, leader

of Image Processing group at MICA central, as my supervisor Thank you for your support and funding of acknowledge to me, thank for mecting to discuss of working every week and your patience during time | worked

and sorry for inconvenience of what I brought to you

1 alao thank to I.e Thị I.an and Thomas Martin, members at MICA,

I couldn’t have done this thesis without your supports ‘hank to both of

you in acknowledge in image processing theory and programming in C! | with the newbie like me

I would like to thank to directors in MICA: Mr Nguyen Trong Ghang, Mr Eric Castclli and Mrs Nguyen Thi Yen who accepted and helped me to have good working environment in MICA ‘Thank to

members in MICA who welcome me to work in MICA as a trainee I

have very good impression in your amiable attitudes and your helps

Finally, I want to thank to my family, my two sisters who often fostered me oven the long distance from homeland Thank to my parent who helped me anytime when | went down and my brother who visited

me sometime to tidy up my room because of my laziness

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Figure I1T-14 Video frame from video sequence |

Figure I-15 Video frame from video scquonec 2

Figure II-16 Key ftarne selection ñom video maosaic

Figure 1LL-17 Key frames is selected from motion graph

igure 1-18 Complicated motion graph from video

Figure ITT-19 cases of vector graph -

Figure I-20 Results of key frame selection

Figure I-21 Hierarchical indexing for CBVIR system

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

11.3.2 Corespond poituls matching

TI4 — Shol Transitions dotzction

11.4.1 Shot cut Detection algorithm

11.42 Shot cut detection description

143 Results and evaluation

TIS Video Indexing

1.5.1 Motion Characterization

11.5.2 Comer-based motion vector

1153 Global Motion calculation

W154 Key frame extraction

1.5.5 Problem in object extraction

iv

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of figures

Figure I-1 Position of video sysizn in MICA ventral

Figure I-1 Two consceutive frames from video sequence

Kigure IL-2 Motion Compensation from MPEG video stream

Figure IL-3 Block diagram of MPEG video encoder

Figure 1-4 Video Ilierarchical Structure

Figure If-5 Connon dircotions of moving videu camera

Figure IL-6 Common rotation and zoom of stationary video camera

+igure I-? CVBIR common system diagram

Vigure 11-8 Classification of video modeling technique Level | with video raw data,

Level II with derived or logical features, and Level III with semantic level

igure 11-10 RGB color space (picture source [SEMMLX)} 19 Figure II-11 LISV color space (picture source [SEMMIX]} 19 Figure I-12 Tarmura features and (heir vatucs (a) Coarseness (b) Contrast (¢)

Figure IE13 Effect of Gabor Filter to image Ti alt

Figure I-16 Reduce the number of bits during calculate the histogram

Figure I-17 Cut (a) and (Fade/Dissolve) trom trame difference

Figure I-18 Twin Comparison (picture taken from (IL 4 5)

Figure If-19: Head tracking for determine trajselories

Figure 1-20: The 2D motion trajcetory (third direction is fame time line) 31 Figure I-21 Optical flow (a) two frame fiom video sequence (b) optical flow 33 Higure Il-22 Optical flow filed produced by pan and track, tilt and boom, zoom and

dolly - aA Figure I-23 Motion sogmentation by optical flow - 238 Figure I-24 Local and Global Contextual Information - 2 +igure IU-1 Relation between R and eigenvalues " 1 Migure II-2 Hatris Comer in image with different given comer nmber a AD gue T1L-3 (2) Two frames extracted while camera pan right (b) correspond points

s, drew lines in frame#760 sl Figwe I 4 Results from no shot transition 84

Figure 1U-5 Results from shot cut transition see SS

Figure IT-7 Correspondent points matching mmbers in one video sequence 60 Figure [0-8 Two frames fiom two shots but similar - -

Tigure II-9 Corresponent points in video sequence 3

Kigure I-10 Frame sequence from video sequence 3

igure 11-11 Keep motion vectors by given threshold for magnitđes 2 65 Figure I1T-12 8 used dircelions for standardizing veclor directions 66 Figure I-13 Some consecutive frames fữom pan ripht shot se 86

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Figure I1T-14 Video frame from video sequence |

Figure I-15 Video frame from video scquonec 2

Figure II-16 Key ftarne selection ñom video maosaic

Figure 1LL-17 Key frames is selected from motion graph

igure 1-18 Complicated motion graph from video

Figure ITT-19 cases of vector graph -

Figure I-20 Results of key frame selection

Figure I-21 Hierarchical indexing for CBVIR system

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of tables

Table | Tast dala used for shot cul algorithm

Table 2 Shot detsvtion result from lest data

‘Table 3 Four types of detection an algorithm can make

‘Table 4 Vector directions rule

Table 5 Calculating global motion fiom set of comer-based vector

Table 6 Vidzo sequence for global motion

Table 7 Three table of motion vectors for video sequence 1, 2 and 6

‘Table 8 Global motion from video sequence 3

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of abbreviations

CD: Compact Disk

DVD: Digital Versatile Disk

MPEG: Moving Pictures Experts Group

CBVIR: Content Based Video Indexing and Retrieval

CBIR: Content Based Indexing and Retrieval

IEC: International Llectro-technical Commission

DCT Discrete Cosine Transform

JPEG: Joint Photographic Experts Group

IDC: Inverse Discrete Cosine Transform

GOP Group of Piclurcs

Tham Quang lai

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of abbreviations

CD: Compact Disk

DVD: Digital Versatile Disk

MPEG: Moving Pictures Experts Group

CBVIR: Content Based Video Indexing and Retrieval

CBIR: Content Based Indexing and Retrieval

IEC: International Llectro-technical Commission

DCT Discrete Cosine Transform

JPEG: Joint Photographic Experts Group

IDC: Inverse Discrete Cosine Transform

GOP Group of Piclurcs

Tham Quang lai

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Acknowledgements

This work is a part in Multimedia Information Communication

Application (MICA) research central

First of all, I would like to thank to Dr Alain Boucher, IT lecturer

at Institut de la Francophonie pour I’ Informatique (JF1), Vietnam, leader

of Image Processing group at MICA central, as my supervisor Thank you for your support and funding of acknowledge to me, thank for mecting to discuss of working every week and your patience during time | worked

and sorry for inconvenience of what I brought to you

1 alao thank to I.e Thị I.an and Thomas Martin, members at MICA,

I couldn’t have done this thesis without your supports ‘hank to both of

you in acknowledge in image processing theory and programming in C! | with the newbie like me

I would like to thank to directors in MICA: Mr Nguyen Trong Ghang, Mr Eric Castclli and Mrs Nguyen Thi Yen who accepted and helped me to have good working environment in MICA ‘Thank to

members in MICA who welcome me to work in MICA as a trainee I

have very good impression in your amiable attitudes and your helps

Finally, I want to thank to my family, my two sisters who often fostered me oven the long distance from homeland Thank to my parent who helped me anytime when | went down and my brother who visited

me sometime to tidy up my room because of my laziness

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Figure I1T-14 Video frame from video sequence |

Figure I-15 Video frame from video scquonec 2

Figure II-16 Key ftarne selection ñom video maosaic

Figure 1LL-17 Key frames is selected from motion graph

igure 1-18 Complicated motion graph from video

Figure ITT-19 cases of vector graph -

Figure I-20 Results of key frame selection

Figure I-21 Hierarchical indexing for CBVIR system

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

11.3.2 Corespond poituls matching

TI4 — Shol Transitions dotzction

11.4.1 Shot cut Detection algorithm

11.42 Shot cut detection description

143 Results and evaluation

TIS Video Indexing

1.5.1 Motion Characterization

11.5.2 Comer-based motion vector

1153 Global Motion calculation

W154 Key frame extraction

1.5.5 Problem in object extraction

iv

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of abbreviations

CD: Compact Disk

DVD: Digital Versatile Disk

MPEG: Moving Pictures Experts Group

CBVIR: Content Based Video Indexing and Retrieval

CBIR: Content Based Indexing and Retrieval

IEC: International Llectro-technical Commission

DCT Discrete Cosine Transform

JPEG: Joint Photographic Experts Group

IDC: Inverse Discrete Cosine Transform

GOP Group of Piclurcs

Tham Quang lai

List of abbreviations v List of figures - - - cet Lisl of tables

Chapter 1 Introduction

L1 Content-based Video Indexing and Retrieval (CB VIR)

12 Aims and Objectives

112 Content-based video indexing and retrieval system

IL2.1 Video sequance structure

112.2, Video data classification,

112.3 Camera operations -

11.2.4, Introduction lo CRVIR systom

112.5 CBVIR Archilecture

3 Features Extraction

I4 Structure Analysi:

TIA1 Shot Transitions classification

1142 Shot Netzetion Teelmiques

1.6.1 Introduction to Video Abstraction

1162 Key frame extraction: -

1163 Video Abstraction - - -

TL7 Vidco Iudexing, Refricval, and Browsing - a AB U8 Thesis Scope 44 Chapter IIL Video Indexing by Camera motions using Comer-based motion vector

TL2 Video and Image Parsing in MICA

HIL2.1 MPEG2 Video Decoder

11.2.3 Video and Image Processing akrry Combination so AB

HL3.1 Hamis Corner points deteotor co 47

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Acknowledgements

This work is a part in Multimedia Information Communication

Application (MICA) research central

First of all, I would like to thank to Dr Alain Boucher, IT lecturer

at Institut de la Francophonie pour I’ Informatique (JF1), Vietnam, leader

of Image Processing group at MICA central, as my supervisor Thank you for your support and funding of acknowledge to me, thank for mecting to discuss of working every week and your patience during time | worked

and sorry for inconvenience of what I brought to you

1 alao thank to I.e Thị I.an and Thomas Martin, members at MICA,

I couldn’t have done this thesis without your supports ‘hank to both of

you in acknowledge in image processing theory and programming in C! | with the newbie like me

I would like to thank to directors in MICA: Mr Nguyen Trong Ghang, Mr Eric Castclli and Mrs Nguyen Thi Yen who accepted and helped me to have good working environment in MICA ‘Thank to

members in MICA who welcome me to work in MICA as a trainee I

have very good impression in your amiable attitudes and your helps

Finally, I want to thank to my family, my two sisters who often fostered me oven the long distance from homeland Thank to my parent who helped me anytime when | went down and my brother who visited

me sometime to tidy up my room because of my laziness

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of tables

Table | Tast dala used for shot cul algorithm

Table 2 Shot detsvtion result from lest data

‘Table 3 Four types of detection an algorithm can make

‘Table 4 Vector directions rule

Table 5 Calculating global motion fiom set of comer-based vector

Table 6 Vidzo sequence for global motion

Table 7 Three table of motion vectors for video sequence 1, 2 and 6

‘Table 8 Global motion from video sequence 3

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

11.3.2 Corespond poituls matching

TI4 — Shol Transitions dotzction

11.4.1 Shot cut Detection algorithm

11.42 Shot cut detection description

143 Results and evaluation

TIS Video Indexing

1.5.1 Motion Characterization

11.5.2 Comer-based motion vector

1153 Global Motion calculation

W154 Key frame extraction

1.5.5 Problem in object extraction

iv

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of tables

Table | Tast dala used for shot cul algorithm

Table 2 Shot detsvtion result from lest data

‘Table 3 Four types of detection an algorithm can make

‘Table 4 Vector directions rule

Table 5 Calculating global motion fiom set of comer-based vector

Table 6 Vidzo sequence for global motion

Table 7 Three table of motion vectors for video sequence 1, 2 and 6

‘Table 8 Global motion from video sequence 3

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of abbreviations

CD: Compact Disk

DVD: Digital Versatile Disk

MPEG: Moving Pictures Experts Group

CBVIR: Content Based Video Indexing and Retrieval

CBIR: Content Based Indexing and Retrieval

IEC: International Llectro-technical Commission

DCT Discrete Cosine Transform

JPEG: Joint Photographic Experts Group

IDC: Inverse Discrete Cosine Transform

GOP Group of Piclurcs

Tham Quang lai

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of figures

Figure I-1 Position of video sysizn in MICA ventral

Figure I-1 Two consceutive frames from video sequence

Kigure IL-2 Motion Compensation from MPEG video stream

Figure IL-3 Block diagram of MPEG video encoder

Figure 1-4 Video Ilierarchical Structure

Figure If-5 Connon dircotions of moving videu camera

Figure IL-6 Common rotation and zoom of stationary video camera

+igure I-? CVBIR common system diagram

Vigure 11-8 Classification of video modeling technique Level | with video raw data,

Level II with derived or logical features, and Level III with semantic level

igure 11-10 RGB color space (picture source [SEMMLX)} 19 Figure II-11 LISV color space (picture source [SEMMIX]} 19 Figure I-12 Tarmura features and (heir vatucs (a) Coarseness (b) Contrast (¢)

Figure IE13 Effect of Gabor Filter to image Ti alt

Figure I-16 Reduce the number of bits during calculate the histogram

Figure I-17 Cut (a) and (Fade/Dissolve) trom trame difference

Figure I-18 Twin Comparison (picture taken from (IL 4 5)

Figure If-19: Head tracking for determine trajselories

Figure 1-20: The 2D motion trajcetory (third direction is fame time line) 31 Figure I-21 Optical flow (a) two frame fiom video sequence (b) optical flow 33 Higure Il-22 Optical flow filed produced by pan and track, tilt and boom, zoom and

dolly - aA Figure I-23 Motion sogmentation by optical flow - 238 Figure I-24 Local and Global Contextual Information - 2 +igure IU-1 Relation between R and eigenvalues " 1 Migure II-2 Hatris Comer in image with different given comer nmber a AD gue T1L-3 (2) Two frames extracted while camera pan right (b) correspond points

s, drew lines in frame#760 sl Figwe I 4 Results from no shot transition 84

Figure 1U-5 Results from shot cut transition see SS

Figure IT-7 Correspondent points matching mmbers in one video sequence 60 Figure [0-8 Two frames fiom two shots but similar - -

Tigure II-9 Corresponent points in video sequence 3

Kigure I-10 Frame sequence from video sequence 3

igure 11-11 Keep motion vectors by given threshold for magnitđes 2 65 Figure I1T-12 8 used dircelions for standardizing veclor directions 66 Figure I-13 Some consecutive frames fữom pan ripht shot se 86

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

11.3.2 Corespond poituls matching

TI4 — Shol Transitions dotzction

11.4.1 Shot cut Detection algorithm

11.42 Shot cut detection description

143 Results and evaluation

TIS Video Indexing

1.5.1 Motion Characterization

11.5.2 Comer-based motion vector

1153 Global Motion calculation

W154 Key frame extraction

1.5.5 Problem in object extraction

iv

List of abbreviations v List of figures - - - cet Lisl of tables

Chapter 1 Introduction

L1 Content-based Video Indexing and Retrieval (CB VIR)

12 Aims and Objectives

112 Content-based video indexing and retrieval system

IL2.1 Video sequance structure

112.2, Video data classification,

112.3 Camera operations -

11.2.4, Introduction lo CRVIR systom

112.5 CBVIR Archilecture

3 Features Extraction

I4 Structure Analysi:

TIA1 Shot Transitions classification

1142 Shot Netzetion Teelmiques

1.6.1 Introduction to Video Abstraction

1162 Key frame extraction: -

1163 Video Abstraction - - -

TL7 Vidco Iudexing, Refricval, and Browsing - a AB U8 Thesis Scope 44 Chapter IIL Video Indexing by Camera motions using Comer-based motion vector

TL2 Video and Image Parsing in MICA

HIL2.1 MPEG2 Video Decoder

11.2.3 Video and Image Processing akrry Combination so AB

HL3.1 Hamis Corner points deteotor co 47

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Acknowledgements

This work is a part in Multimedia Information Communication

Application (MICA) research central

First of all, I would like to thank to Dr Alain Boucher, IT lecturer

at Institut de la Francophonie pour I’ Informatique (JF1), Vietnam, leader

of Image Processing group at MICA central, as my supervisor Thank you for your support and funding of acknowledge to me, thank for mecting to discuss of working every week and your patience during time | worked

and sorry for inconvenience of what I brought to you

1 alao thank to I.e Thị I.an and Thomas Martin, members at MICA,

I couldn’t have done this thesis without your supports ‘hank to both of

you in acknowledge in image processing theory and programming in C! | with the newbie like me

I would like to thank to directors in MICA: Mr Nguyen Trong Ghang, Mr Eric Castclli and Mrs Nguyen Thi Yen who accepted and helped me to have good working environment in MICA ‘Thank to

members in MICA who welcome me to work in MICA as a trainee I

have very good impression in your amiable attitudes and your helps

Finally, I want to thank to my family, my two sisters who often fostered me oven the long distance from homeland Thank to my parent who helped me anytime when | went down and my brother who visited

me sometime to tidy up my room because of my laziness

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Acknowledgements

This work is a part in Multimedia Information Communication

Application (MICA) research central

First of all, I would like to thank to Dr Alain Boucher, IT lecturer

at Institut de la Francophonie pour I’ Informatique (JF1), Vietnam, leader

of Image Processing group at MICA central, as my supervisor Thank you for your support and funding of acknowledge to me, thank for mecting to discuss of working every week and your patience during time | worked

and sorry for inconvenience of what I brought to you

1 alao thank to I.e Thị I.an and Thomas Martin, members at MICA,

I couldn’t have done this thesis without your supports ‘hank to both of

you in acknowledge in image processing theory and programming in C! | with the newbie like me

I would like to thank to directors in MICA: Mr Nguyen Trong Ghang, Mr Eric Castclli and Mrs Nguyen Thi Yen who accepted and helped me to have good working environment in MICA ‘Thank to

members in MICA who welcome me to work in MICA as a trainee I

have very good impression in your amiable attitudes and your helps

Finally, I want to thank to my family, my two sisters who often fostered me oven the long distance from homeland Thank to my parent who helped me anytime when | went down and my brother who visited

me sometime to tidy up my room because of my laziness

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of figures

Figure I-1 Position of video sysizn in MICA ventral

Figure I-1 Two consceutive frames from video sequence

Kigure IL-2 Motion Compensation from MPEG video stream

Figure IL-3 Block diagram of MPEG video encoder

Figure 1-4 Video Ilierarchical Structure

Figure If-5 Connon dircotions of moving videu camera

Figure IL-6 Common rotation and zoom of stationary video camera

+igure I-? CVBIR common system diagram

Vigure 11-8 Classification of video modeling technique Level | with video raw data,

Level II with derived or logical features, and Level III with semantic level

igure 11-10 RGB color space (picture source [SEMMLX)} 19 Figure II-11 LISV color space (picture source [SEMMIX]} 19 Figure I-12 Tarmura features and (heir vatucs (a) Coarseness (b) Contrast (¢)

Figure IE13 Effect of Gabor Filter to image Ti alt

Figure I-16 Reduce the number of bits during calculate the histogram

Figure I-17 Cut (a) and (Fade/Dissolve) trom trame difference

Figure I-18 Twin Comparison (picture taken from (IL 4 5)

Figure If-19: Head tracking for determine trajselories

Figure 1-20: The 2D motion trajcetory (third direction is fame time line) 31 Figure I-21 Optical flow (a) two frame fiom video sequence (b) optical flow 33 Higure Il-22 Optical flow filed produced by pan and track, tilt and boom, zoom and

dolly - aA Figure I-23 Motion sogmentation by optical flow - 238 Figure I-24 Local and Global Contextual Information - 2 +igure IU-1 Relation between R and eigenvalues " 1 Migure II-2 Hatris Comer in image with different given comer nmber a AD gue T1L-3 (2) Two frames extracted while camera pan right (b) correspond points

s, drew lines in frame#760 sl Figwe I 4 Results from no shot transition 84

Figure 1U-5 Results from shot cut transition see SS

Figure IT-7 Correspondent points matching mmbers in one video sequence 60 Figure [0-8 Two frames fiom two shots but similar - -

Tigure II-9 Corresponent points in video sequence 3

Kigure I-10 Frame sequence from video sequence 3

igure 11-11 Keep motion vectors by given threshold for magnitđes 2 65 Figure I1T-12 8 used dircelions for standardizing veclor directions 66 Figure I-13 Some consecutive frames fữom pan ripht shot se 86

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of figures

Figure I-1 Position of video sysizn in MICA ventral

Figure I-1 Two consceutive frames from video sequence

Kigure IL-2 Motion Compensation from MPEG video stream

Figure IL-3 Block diagram of MPEG video encoder

Figure 1-4 Video Ilierarchical Structure

Figure If-5 Connon dircotions of moving videu camera

Figure IL-6 Common rotation and zoom of stationary video camera

+igure I-? CVBIR common system diagram

Vigure 11-8 Classification of video modeling technique Level | with video raw data,

Level II with derived or logical features, and Level III with semantic level

igure 11-10 RGB color space (picture source [SEMMLX)} 19 Figure II-11 LISV color space (picture source [SEMMIX]} 19 Figure I-12 Tarmura features and (heir vatucs (a) Coarseness (b) Contrast (¢)

Figure IE13 Effect of Gabor Filter to image Ti alt

Figure I-16 Reduce the number of bits during calculate the histogram

Figure I-17 Cut (a) and (Fade/Dissolve) trom trame difference

Figure I-18 Twin Comparison (picture taken from (IL 4 5)

Figure If-19: Head tracking for determine trajselories

Figure 1-20: The 2D motion trajcetory (third direction is fame time line) 31 Figure I-21 Optical flow (a) two frame fiom video sequence (b) optical flow 33 Higure Il-22 Optical flow filed produced by pan and track, tilt and boom, zoom and

dolly - aA Figure I-23 Motion sogmentation by optical flow - 238 Figure I-24 Local and Global Contextual Information - 2 +igure IU-1 Relation between R and eigenvalues " 1 Migure II-2 Hatris Comer in image with different given comer nmber a AD gue T1L-3 (2) Two frames extracted while camera pan right (b) correspond points

s, drew lines in frame#760 sl Figwe I 4 Results from no shot transition 84

Figure 1U-5 Results from shot cut transition see SS

Figure IT-7 Correspondent points matching mmbers in one video sequence 60 Figure [0-8 Two frames fiom two shots but similar - -

Tigure II-9 Corresponent points in video sequence 3

Kigure I-10 Frame sequence from video sequence 3

igure 11-11 Keep motion vectors by given threshold for magnitđes 2 65 Figure I1T-12 8 used dircelions for standardizing veclor directions 66 Figure I-13 Some consecutive frames fữom pan ripht shot se 86

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Acknowledgements

This work is a part in Multimedia Information Communication

Application (MICA) research central

First of all, I would like to thank to Dr Alain Boucher, IT lecturer

at Institut de la Francophonie pour I’ Informatique (JF1), Vietnam, leader

of Image Processing group at MICA central, as my supervisor Thank you for your support and funding of acknowledge to me, thank for mecting to discuss of working every week and your patience during time | worked

and sorry for inconvenience of what I brought to you

1 alao thank to I.e Thị I.an and Thomas Martin, members at MICA,

I couldn’t have done this thesis without your supports ‘hank to both of

you in acknowledge in image processing theory and programming in C! | with the newbie like me

I would like to thank to directors in MICA: Mr Nguyen Trong Ghang, Mr Eric Castclli and Mrs Nguyen Thi Yen who accepted and helped me to have good working environment in MICA ‘Thank to

members in MICA who welcome me to work in MICA as a trainee I

have very good impression in your amiable attitudes and your helps

Finally, I want to thank to my family, my two sisters who often fostered me oven the long distance from homeland Thank to my parent who helped me anytime when | went down and my brother who visited

me sometime to tidy up my room because of my laziness

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Acknowledgements

This work is a part in Multimedia Information Communication

Application (MICA) research central

First of all, I would like to thank to Dr Alain Boucher, IT lecturer

at Institut de la Francophonie pour I’ Informatique (JF1), Vietnam, leader

of Image Processing group at MICA central, as my supervisor Thank you for your support and funding of acknowledge to me, thank for mecting to discuss of working every week and your patience during time | worked

and sorry for inconvenience of what I brought to you

1 alao thank to I.e Thị I.an and Thomas Martin, members at MICA,

I couldn’t have done this thesis without your supports ‘hank to both of

you in acknowledge in image processing theory and programming in C! | with the newbie like me

I would like to thank to directors in MICA: Mr Nguyen Trong Ghang, Mr Eric Castclli and Mrs Nguyen Thi Yen who accepted and helped me to have good working environment in MICA ‘Thank to

members in MICA who welcome me to work in MICA as a trainee I

have very good impression in your amiable attitudes and your helps

Finally, I want to thank to my family, my two sisters who often fostered me oven the long distance from homeland Thank to my parent who helped me anytime when | went down and my brother who visited

me sometime to tidy up my room because of my laziness

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of figures

Figure I-1 Position of video sysizn in MICA ventral

Figure I-1 Two consceutive frames from video sequence

Kigure IL-2 Motion Compensation from MPEG video stream

Figure IL-3 Block diagram of MPEG video encoder

Figure 1-4 Video Ilierarchical Structure

Figure If-5 Connon dircotions of moving videu camera

Figure IL-6 Common rotation and zoom of stationary video camera

+igure I-? CVBIR common system diagram

Vigure 11-8 Classification of video modeling technique Level | with video raw data,

Level II with derived or logical features, and Level III with semantic level

igure 11-10 RGB color space (picture source [SEMMLX)} 19 Figure II-11 LISV color space (picture source [SEMMIX]} 19 Figure I-12 Tarmura features and (heir vatucs (a) Coarseness (b) Contrast (¢)

Figure IE13 Effect of Gabor Filter to image Ti alt

Figure I-16 Reduce the number of bits during calculate the histogram

Figure I-17 Cut (a) and (Fade/Dissolve) trom trame difference

Figure I-18 Twin Comparison (picture taken from (IL 4 5)

Figure If-19: Head tracking for determine trajselories

Figure 1-20: The 2D motion trajcetory (third direction is fame time line) 31 Figure I-21 Optical flow (a) two frame fiom video sequence (b) optical flow 33 Higure Il-22 Optical flow filed produced by pan and track, tilt and boom, zoom and

dolly - aA Figure I-23 Motion sogmentation by optical flow - 238 Figure I-24 Local and Global Contextual Information - 2 +igure IU-1 Relation between R and eigenvalues " 1 Migure II-2 Hatris Comer in image with different given comer nmber a AD gue T1L-3 (2) Two frames extracted while camera pan right (b) correspond points

s, drew lines in frame#760 sl Figwe I 4 Results from no shot transition 84

Figure 1U-5 Results from shot cut transition see SS

Figure IT-7 Correspondent points matching mmbers in one video sequence 60 Figure [0-8 Two frames fiom two shots but similar - -

Tigure II-9 Corresponent points in video sequence 3

Kigure I-10 Frame sequence from video sequence 3

igure 11-11 Keep motion vectors by given threshold for magnitđes 2 65 Figure I1T-12 8 used dircelions for standardizing veclor directions 66 Figure I-13 Some consecutive frames fữom pan ripht shot se 86

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of tables

Table | Tast dala used for shot cul algorithm

Table 2 Shot detsvtion result from lest data

‘Table 3 Four types of detection an algorithm can make

‘Table 4 Vector directions rule

Table 5 Calculating global motion fiom set of comer-based vector

Table 6 Vidzo sequence for global motion

Table 7 Three table of motion vectors for video sequence 1, 2 and 6

‘Table 8 Global motion from video sequence 3

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

Figure I1T-14 Video frame from video sequence |

Figure I-15 Video frame from video scquonec 2

Figure II-16 Key ftarne selection ñom video maosaic

Figure 1LL-17 Key frames is selected from motion graph

igure 1-18 Complicated motion graph from video

Figure ITT-19 cases of vector graph -

Figure I-20 Results of key frame selection

Figure I-21 Hierarchical indexing for CBVIR system

Thesis for Degree of duster - Content-based videu indexing oud retrrevial

List of tables

Table | Tast dala used for shot cul algorithm

Table 2 Shot detsvtion result from lest data

‘Table 3 Four types of detection an algorithm can make

‘Table 4 Vector directions rule

Table 5 Calculating global motion fiom set of comer-based vector

Table 6 Vidzo sequence for global motion

Table 7 Three table of motion vectors for video sequence 1, 2 and 6

‘Table 8 Global motion from video sequence 3