Multimedia Database Management Systems.pdf

http //www netlibrary com/nlreader/nlreader dll?bookid=40411&fi Page iii Multimedia Database Management Systems Guojun Lu Page 1 of 1Document 2/29/2004http //www netlibrary com/nlreader/nlreader dll?b[.]

Multimedia Database Management Systems

Guojun Lu

Page iv

Library of Congress Cataloging-in-Publication Data

Lu, Guojun, 1963-

Multimedia database management systems / Guojun Lu

P cm — (Artech House computing library)

Includes bibliographical references and index

ISBN 1-89006-342-7 (alk paper)

1 Mulitimedia systems 2 Database management I Title.

QA76.575.L84 1999 99-41773

006.7—dc21 CIP

British Library Cataloguing in Publication Data

Lu, Guojun, 1963-

Multimedia database management systems — (Artech House computing library)

1 Multimedia systems 2 Database management

I Title

005.7'4

ISBN 0-89006-342-7

Cover design by Elaine K Donnelly

© 1999 ARTECH HOUSE, INC.

International Standard Book Number: 0-89006-342-7

Library in Congress Catalog Card Number: 99-41773

10 9 8 7 6 5 4 3 2 1

To Fengxia, James, and Colin

1.1.5 Feature Extraction, Content Representation, and

Indexing

3

1.2.1 Proliferation of Multimedia Data and Its

1.2.3 IR Systems and Their Role in Multimedia Retrieval 7

1.2.4 Integrated Approach to Multimedia Information

Indexing and Retrieval

2.4.1 Basic Characteristics of Audio Signal 18

Page viii

Determining the Number of Quantization Levels 22

2.4.3 Musical Instrument Digital Interface (MIDI) 24

The JPEG-Still Image Compression Standard 33

2.7 Standards for Composite Multimedia Documents 43

2.8 Major Characteristics and Requirements of Multimedia

Data and Applications

44

2.8.2 Semantic Structure of Multimedia Information 46

2.8.4 Temporal and Spatial Relationships among Related

Media

47

2.8.5 Subjectiveness and Fuzziness of the Meaning of 47

4.3 Automatic Text Document Indexing and Boolean

Retrieval Model

75

Extensions of the Inverted File Operation 77

4.3.3 Term Operations and Automatic Indexing 78

4.3.4 Summary of Automatic Document Indexing 80

Page x

4.10.3.2 The Number of WWW Documents Is Large 94

4.10.3.3 WWW Documents Are Dynamic and

Heterogeneous

95

4.10.3.5 WWW Search Engines Are Heavily Used 96

4.10.4 General Structure of WWW Search Engines 96

4.10.5.1 Architecture Overview of Google 97

5.2.1 Features Derived in the Time Domain 106

5.2.2 Features Derived from the Frequency Domain 108

5.3.1 Main Characteristics of Different Types of Sound 113

5.4.1.2 Techniques Based on Dynamic Time Warping 118

5.5.1 Indexing and Retrieval of Structured Music and

Sound Effects

122

5.5.2 Indexing and Retrieval of Sample-Based Music 123

Music Retrieval Based on a Set of Features 123

5.6 Multimedia Information Indexing and Retrieval Using

Relationships between Audio and Other Media

125

6.2 Different Approaches to Image Indexing and Retrieval 132

6.4 Color-Based Image Indexing and Retrieval Techniques 133

6.4.1 The Basic Color-Based Image Retrieval Technique 133

6.4.2 Improvements to the Basic Technique 134

6.4.2.1 Making Use of Similarity among Colors 135

6.4.2.2 Making Use of Spatial Relationships among

Effects of Different Image Representations on Retrieval Performance

139

6.5.1 Definitions of Common Terms and Some Simple

Shape Measurements

142

6.5.7 Region-Based Shape Representation and Similarity

6.5.7.8 Summary of Index and Retrieval Processes 152

6.7 Image Indexing and Retrieval Based on Compressed

6.8 Other Image Indexing and Retrieval Techniques 159

6.8.1 Image Retrieval Based on Model-Based

Compression

159

6.8.2 Image Retrieval Based on Spatial Relationship 159

6.9 Integrated Image Indexing and Retrieval Techniques 159

6.9.2 Virage Image Search Engine 160

Relationships between CIE XYZ and Other Color

Spaces

167

7.2 Overview of Shot-Based Video Indexing and Retrieval 180

7.3.2 Detecting Shot Boundaries with Gradual Change 182

7.4.1 Indexing and Retrieval Based on r Frames of Video

7.4.3 Indexing and Retrieval Based on Objects 189

7.4.4 Indexing and Retrieval Based on Metadata 190

7.4.5 Indexing and Retrieval Based on Annotation 190

7.4.6 Integrated Approach to Video Indexing and

Retrieval

190

7.5 Effective Video Representation and Abstraction 191

8.2 Integrated Indexing and Retrieval Techniques 202

8.2.1 Integrated Audio Indexing and Retrieval 203

8.2.2 Integrated Image Indexing and Retrieval 204

8.2.3 Integrated Video Indexing and Retrieval 204

8.2.4 Merging of Results Obtained Based on Individual

Result Presentation and Relevance Feedback 209

8.5.2 An Integrated WWW Image Search Engine

Developed at Monash University

211

Text-based Image Indexing and Retrieval 212

Color-based Image Indexing and Retrieval 213

Image Retrieval Combining Text and Color-based

9.2 Filtering Processes for Reducing Search Space 224

9.2.1 Filtering with Classification, Structured Attributes,

and Keywords

225

9.2.2 Methods Based on the Triangle Inequality 225

9.2.3 Methods Specific to Color-Histogram-Based

9.5.4 Higher Dimensional MB+ Trees 239

9.8.1 An Overview of the R Tree Structure 242

9.8.2 Search, Insertion and Deletion of Region Objects 244

9.8.3 Search, Insertion and Deletion of Point Data 244

9.8.5 R* Tree, R+ Tree and VAMSplit R Tree 245

10.4 Multimedia Data Storage Devices and Management 261

10.4.1 Multimedia Storage Server Requirements 262

Second-Level RAID: Hamming Code for Error Correction

264

Third-Level RAID: Single Check Disk Per Group 265

Fifth-Level RAID: No Dedicated Check Disk 266

10.4.4 Disk Scheduling and Admission Control 269

10.4.4.1 Traditional Disk-Scheduling Algorithms 270

10.4.4.4 Round-Robin 271

10.4.5.3 QoS Issue Related to User Interactions 275

10.4.6 Server Configuration and Network Connection 276

10.5.1 Processor Architectures for Multimedia 278

10.5.1.2 Dedicated Multimedia Processors 279

10.5.2.3 The Use of Dedicated Multimedia Devices 282

10.5.2.4 Network-Based Multimedia Computer

Architecture

283

10.6.1 Multimedia Operating System Requirements 284

10.6.3 Conventional Time-Sharing Operating Systems and

Incorporation of Real-time Features

285

10.6.4.3 Single-Copy and Zero-Copy Architecture 288

10.6.6.3 Resource Reservation and Policing 291

10.6.6.4 Process Scheduling Disciplines 291

10.6.6.5 QoS Graceful Degradation and Media

10.7.1.2 Efficient Sharing of Network Resources 296

10.7.2.1 What Is ATM? 300

10.7.2.2 B-ISDN Protocol Reference Model 302

10.7.2.3 Why Is ATM Suitable for Multimedia

Communications?

304

10.8.1 Requirements of Multimedia Transport Protocols 306

10.8.1.2 QoS Specification and Guarantee 307

10.8.2 Why Traditional Transport Protocols Are Not

Suitable for Multimedia Communications

10.8.2.6 Suitability of eXpress Transport Protocol 310

10.8.4 Real-Time Transport Protocol (RTP) 315

10.8.5 Other Multimedia Transport Protocols: HeiTP and

10.9.3.3 Measures to Counter Clock Rate Difference 324

10.9.3.4 Measures to Counter Packet-Out-of-Order

11.5 Percentage of Similarity Rankings (PSR) 339

11.6 Suitability of the Common Effectiveness Measurements 339

12.3 Applications of Multimedia Indexing and Retrieval 348

Page xviii

A few years ago, the main focus in multimedia research and development was on multimedia communications and multimedia authoring and presentation Since more and more digital multimedia data in the forms of image, video, and audio are being captured and stored, the focus has shifted in last few years to efficient and effective storage and retrieval of multimedia information A similar situation happened about thirty years ago when more and more alphanumeric data were stored in computer- readable form This led to the development of database management systems

(DBMSs), which are now one of the most popular computer applications and are used

in almost every organization Due to the differences between the characteristics and requirements of alphanumeric data and multimedia data, a traditional DBMS is not capable of effectively handling multimedia data Thus new multimedia indexing and retrieval techniques are required.

This book provides comprehensive coverage on issues and current technologies in multimedia database management It starts with a discussion on the main

characteristics and requirements of multimedia data It then describes general design issues of multimedia database management systems to meet these characteristics and requirements We discuss techniques for the indexing and retrieval of text documents, audio, images, and video The commonality among these techniques is that they all try to extract main features from raw media data, and then try to retrieve items

relevant to the user's query based on the similarity or the distance between the

extracted feature vectors of the stored items and the query As multiple media types usually appear together in multimedia objects or documents and different techniques capture different features or contents of them, we discuss how to integrate various indexing and retrieval techniques for effective retrieval of multimedia documents Since the features extracted from raw media data are usually represented by

multidimensional vectors, it would be very time-consuming to calculate the similarity between the query feature vector and the feature vector of each of the stored items

We discuss various techniques and data structures so that the search and retrieval can

be carried out efficiently.

Multimedia databases are usually accessed remotely oyer a network Multimedia objects identified as relevant to the query must be retrieved from the server and

transmitted to the client for presentation A set of requirements involving factors including delay and delay jitter, called quality of service, must be met to achieve multimedia presentation in a smooth and timely fashion We describe computer

architecture, multimedia storage, operating system, and networking support to meet these requirements.

In traditional DBMSs, the main performance concern is efficiency (how long it takes

to answer a query) In multimedia database management systems (MMDBMSs), efficiency is important too In addition, retrieval effectiveness (ability to retrieve rele-

Page xxvant items and ability to reject irrelevant items) becomes important This is because MMDBMSs retrieve items based on similarity measured using a similarity metric instead of exact match Since it is difficult to extract all features of multimedia items and design a similarity metric that exactly conforms to human judgment, it is likely that some items determined as relevant by the system are actually judged irrelevant by the user and some relevant items are not retrieved Thus we also discuss performance measurement issues in MMDBMSs Finally, we briefly describe current products, application development, and other issues such as security and standardization.

This book is intended for the following groups of readers:

• University students at senior levels and postgraduate students Many universities around the world have started or will start to offer subjects related to multimedia technology and MMDBMSs This book serves as a text for such subjects.

• System designers and developers who want to learn various issues and techniques in developing multimedia database management systems.

• Researchers who want to learn the current developments and new research

Technology of Monash University for their support during the last three years.

Dr Ilias Petrounias of UMIST in Machester, United Kingdom, reviewed the book and provided many helpful comments and suggestions, for which I am most grateful.

I dedicate this book to my wife Fengxia, sons James and Colin, without whose

support and understanding the writing of this book would not have been possible.

Chapter 1—

Introduction

The initial focus in multimedia research and development was on multimedia

communications and multimedia authoring and presentation [1–4] In the past few years, more and more digital multimedia data in the forms of image, video, and audio has been captured and stored There is now a strong research interest in efficient and effective storage and retrieval of multimedia information A similar situation

happened about thirty years ago when more and more alphanumeric data was stored

in computer readable form This led to the development of database management systems (DBMSs), which are now one of the most popular computer applications and are used in almost every organization However, DBMSs cannot handle multimedia data effectively because of the differences between the characteristics of

alphanumeric data and multimedia data Therefore, new multimedia indexing and retrieval techniques are required.

The main purpose of this book is to describe issues and current technologies in

multimedia indexing and retrieval The area of multimedia indexing and retrieval is still in its early development stage But it is expected that multimedia database

management systems (MMDBMSs) will be as popular as current DBMSs in the near future [5–7].

This chapter first defines some important terms and concepts used throughout the book, and explains why traditional DBMS cannot handle multimedia information and why new technologies are required to support efficient and effective multimedia information retrieval It then gives an overview of multimedia indexing and retrieval systems (MIRSs) Some expected capabilities and common applications of MIRSs are presented The chapter concludes with an outline of the organization of the following chapters.

1.1—

Some Important Definitions

To avoid any confusion or misunderstanding, this section provides definitions for some important terms and concepts used in this book.

1.1.1—

Media Types and Multimedia

Media refer to the types of information or types of information representation, such as

alphanumeric data, images, audio, and video There are many ways to classify media Common classifications are based on physical formats and media relationships with time In this book, we classify media based on whether or not there are time

dimensions

Page 2

to them This convention leads to two classes of media: static and dynamic (or time continuous).

Static media do not have a time dimension, and their contents and meanings do not

depend on the presentation time Static media include alphanumeric data, graphics, and still images.

Dynamic media have time dimensions, and their meanings and correctness depend on

the rate at which they are presented Dynamic media include animation, audio, and video These media have their intrinsic unit intervals or rates For example, to convey

a perceptually smooth movement, video must be played back at 25 frames per second (or 30 frames, depending on the video system used) Similarly, when we play back a recorded voice message or music, only one playback rate is natural or sensible

Playback at a slower or faster rate distorts the meaning or the quality of the sound Because these media must be played back continuously at a fixed rate, they are often

called continuous media They are also called isochronous media because of the fixed

relationship between each media unit and time.

Multimedia refers to a collection of media types used together It is implied that at least one media type is not alphanumeric data (i.e., at least one media type is image, audio, or video) In this book, ''multimedia'' is used as an adjective—so we will

specifically say multimedia information, multimedia data, multimedia system,

multimedia communications, multimedia applications, and so forth Multimedia data refers to the computer-readable representation of multiple media types Multimedia information refers to the information conveyed by multiple media types Sometimes, multimedia information and multimedia data are used interchangeably.

We sometimes use multimedia or media item and object to refer to any autonomous entity in an MIRS that can be queried, retrieved, and presented The term "object" may not be properly defined in the technical object-oriented (OO) sense The context should make it clear whether it is used in a general sense or refers to a properly

defined object in an OO approach.

1.1.2—

Databases and DBMSs

In the literature, databases and DBMSs are sometimes used interchangeably In this book, database refers to a collection or repository of data or media items We use DBMS to refer to the entire system that manages the database.

1.1.3—

Text Document Information Retrieval

Automated information retrieval (IR) systems were developed to help manage the huge amount of scientific literature that has been created since the 1940s [8, 9] The main function of an IR system is to store and manage a large number of text