Multimedia Image and Video Processing P1

5.5.2 Postprocessing5.5.3 Shape and Color Analysis5.5.4 Fuzzy Membership Functions5.5.5 Meta-Data Features5.5.6 Experimental Results5.6 Conclusions References 6 Intelligent Multimedia Pr

Guan, Ling et al “Frontmatter”

Multimedia Image and Video Processing

Ed Ling Guan et al.

Boca Raton: CRC Press LLC, 2001

PROCESSING

IMAGE PROCESSING SERIESSeries Editor: Phillip A Laplante

Forthcoming Titles

Adaptive Image Processing: A Computational Intelligence

Perspective

Ling Guan, Hau-San Wong, and Stuart William Perry

Shape Analysis and Classification: Theory and Practice

Luciano da Fontoura Costa and Roberto Marcondes Cesar, Jr

Published Titles

Image and Video Compression for Multimedia Engineering

Yun Q Shi and Huiyang Sun

Boca Raton London New York Washington, D.C.

CRC Press

Edited by

Ling Guan Sun-Yuan Kung

Jan Larsen

MULTIMEDIA

PROCESSING

This book contains information obtained from authentic and highly regarded sources Reprinted material isquoted with permission, and sources are indicated A wide variety of references are listed Reasonable effortshave been made to publish reliable data and information, but the author and the publisher cannot assumeresponsibility for the validity of all materials or for the consequences of their use.

Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic ormechanical, including photocopying, microfilming, and recording, or by any information storage or retrievalsystem, without prior permission in writing from the publisher

All rights reserved Authorization to photocopy items for internal or personal use, or the personal or internaluse of specific clients, may be granted by CRC Press LLC, provided that $.50 per page photocopied is paiddirectly to Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 USA The fee code forusers of the Transactional Reporting Service is ISBN 0-8493-3492-6/01/$0.00+$.50 The fee is subject tochange without notice For organizations that have been granted a photocopy license by the CCC, a separatesystem of payment has been arranged

The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for creatingnew works, or for resale Specific permission must be obtained in writing from CRC Press LLC for suchcopying

Direct all inquiries to CRC Press LLC, 2000 N.W Corporate Blvd., Boca Raton, Florida 33431

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are usedonly for identification and explanation, without intent to infringe

© 2001 by CRC Press LLC

No claim to original U.S Government worksInternational Standard Book Number 0-8493-3492-6Library of Congress Card Number 00-030341Printed in the United States of America 1 2 3 4 5 6 7 8 9 0

Printed on acid-free paper

Library of Congress Cataloging-in-Publication Data

Multimedia image and video processing / edited by Ling Guan, Sun-Yuan Kung, Jan Larsen

p cm

Includes bibliographical references and index

ISBN 0-8493-3492-6 (alk.)

1 Multimedia systems 2 Image processing—Digital techniques I Guan, Ling II

Kung, S.Y (Sun Yuan) III Larsen, Jan

QA76.575 2000

1 Emerging Standards for Multimedia Applications

Tsuhan Chen

1.1 Introduction1.2 Standards1.3 Fundamentals of Video Coding1.3.1 Transform Coding1.3.2 Motion Compensation1.3.3 Summary

1.4 Emerging Video and Multimedia Standards1.4.1 H.263

1.4.2 H.26L1.4.3 MPEG-41.4.4 MPEG-71.5 Standards for Multimedia Communication1.6 Conclusion

2.2.2 Existing Approximation Methods2.2.3 Constant Denominator Method2.2.4 Simulation Results

2.2.5 Sprite Warping Algorithm2.3 Architecture for Sprite Warping2.3.1 Implementation Issues2.3.2 Memory Bandwidth Reduction2.3.3 Architecture

2.4 ConclusionReferences

3 Application-Specific Multimedia Processor Architecture

Yu Hen Hu and Surin Kittitornkun

3.1 Introduction3.1.1 Requirements of Multimedia Signal Processing (MSP) Hardware3.1.2 Strategies: Matching Micro-Architecture and Algorithm

3.2 Systolic Array Structure Micro-Architecture3.2.1 Systolic Array Design Methodology3.2.2 Array Structures for Motion Estimation3.3 Dedicated Micro-Architecture

3.3.1 Design Methodologies for Dedicated Micro-Architecture3.3.2 Feed-Forward Direct Synthesis: Fast Discrete Cosine Transform (DCT)3.3.3 Feedback Direct Synthesis: Huffman Coding

3.4 Concluding RemarksReferences

4 Superresolution of Images with Learned Multiple Reconstruction Kernels

Frank M Candocia and Jose C Principe

4.1 Introduction4.2 An Approach to Superresolution4.2.1 Comments and Observations4.2.2 Finding Bases for Image Representation4.2.3 Description of the Methodology4.3 Image Acquisition Model

4.4 Relating Kernel-Based Approaches4.4.1 Single Kernel

4.4.2 Family of Kernels4.5 Description of the Superresolution Architecture4.5.1 The Training Data

4.5.2 Clustering of Data4.5.3 Neighborhood Association4.5.4 Superresolving Images4.6 Results

4.7 Issues and Notes4.8 ConclusionsReferences

5 Image Processing Techniques for Multimedia Processing

N Herodotou, K.N Plataniotis, and A.N Venetsanopoulos

5.1 Introduction5.2 Color in Multimedia Processing5.3 Color Image Filtering

5.3.1 Fuzzy Multichannel Filters5.3.2 The Membership Functions5.3.3 A Combined Fuzzy Directional and Fuzzy Median Filter5.3.4 Application to Color Images

5.4 Color Image Segmentation5.4.1 Histogram Thresholding5.4.2 Postprocessing and Region Merging5.4.3 Experimental Results

5.5 Facial Image Segmentation5.5.1 Extraction of Skin-Tone Regions

5.5.2 Postprocessing5.5.3 Shape and Color Analysis5.5.4 Fuzzy Membership Functions5.5.5 Meta-Data Features

5.5.6 Experimental Results5.6 Conclusions

References

6 Intelligent Multimedia Processing

Ling Guan, Sun-Yuan Kung, and Jenq-Neng Hwang

6.1 Introduction6.1.1 Neural Networks and Multimedia Processing6.1.2 Focal Technical Issues Addressed in the Chapter6.1.3 Organization of the Chapter

6.2 Useful Neural Network Approaches to Multimedia Data Representation, sification, and Fusion

Clas-6.2.1 Multimedia Data Representation6.2.2 Multimedia Data Detection and Classification6.2.3 Hierarchical Fuzzy Neural Networks as Linear Fusion Networks6.2.4 Temporal Models for Multimodal Conversion and Synchronization6.3 Neural Networks for IMP Applications

6.3.1 Image Visualization and Segmentation6.3.2 Personal Authentication and Recognition6.3.3 Audio-to-Visual Conversion and Synchronization6.3.4 Image and Video Retrieval, Browsing, and Content-Based Indexing6.3.5 Interactive Human–Computer Vision

6.4 Open Issues, Future Research Directions, and ConclusionsReferences

7 On Independent Component Analysis for Multimedia Signals

Lars Kai Hansen, Jan Larsen, and Thomas Kolenda

7.1 Background7.2 Principal and Independent Component Analysis7.3 Likelihood Framework for Independent Component Analysis7.3.1 Generalization and the Bias-Variance Dilemma7.3.2 Noisy Mixing of White Sources

7.3.3 Separation Based on Time Correlation7.3.4 Likelihood

7.4 Separation of Sound Signals7.4.1 Sound Separation using PCA7.4.2 Sound Separation using Molgedey–Schuster ICA7.4.3 Sound Separation using Bell–Sejnowski ICA7.4.4 Comparison

7.5 Separation of Image Mixtures7.5.1 Image Segmentation using PCA7.5.2 Image Segmentation using Molgedey–Schuster ICA7.5.3 Discussion

7.6 ICA for Text Representation7.6.1 Text Analysis7.6.2 Latent Semantic Analysis — PCA7.6.3 Latent Semantic Analysis — ICA

7.7 ConclusionAcknowledgmentAppendix AReferences

8 Image Analysis and Graphics for Multimedia Presentation

Tülay Adali and Yue Wang

8.1 Introduction8.2 Image Analysis8.2.1 Pixel Modeling8.2.2 Model Identification8.2.3 Context Modeling8.2.4 Applications8.3 Graphics Modeling8.3.1 Surface Reconstruction8.3.2 Physical Deformable Models8.3.3 Deformable Surface–Spine Models8.3.4 Numerical Implementation8.3.5 Applications

References

9 Combined Motion Estimation and Transform Coding in Compressed Domain

Ut-Va Koc and K.J Ray Liu

9.1 Introduction9.2 Fully DCT-Based Motion-Compensated Video Coder Structure9.3 DCT Pseudo-Phase Techniques

9.4 DCT-Based Motion Estimation9.4.1 The DXT-ME Algorithm9.4.2 Computational Issues and Complexity9.4.3 Preprocessing

9.4.4 Adaptive Overlapping Approach9.4.5 Simulation Results

9.5 Subpixel DCT Pseudo-Phase Techniques9.5.1 Subpel Sinusoidal Orthogonality Principles9.6 DCT-Based Subpixel Motion Estimation

9.6.1 DCT-Based Half-Pel Motion Estimation Algorithm (HDXT-ME)9.6.2 DCT-Based Quarter-Pel Motion Estimation Algorithm (QDXT-MEand Q4DXT-ME)

9.6.3 Simulation Results9.7 DCT-Based Motion Compensation9.7.1 Integer-Pel DCT-Based Motion Compensation9.7.2 Subpixel DCT-Based Motion Compensation9.7.3 Simulation

9.8 ConclusionReferences

10 Object-Based Analysis–Synthesis Coding Based on Moving 3D Objects

Jörn Ostermann

10.1 Introduction10.2 Object-Based Analysis–Synthesis Coding10.3 Source Models for OBASC

10.3.1 Camera Model10.3.2 Scene Model10.3.3 Illumination Model10.3.4 Object Model10.4 Image Analysis for 3D Object Models10.4.1 Overview

10.4.2 Motion Estimation for R3D10.4.3 MF Objects

10.5 Optimization of Parameter Coding for R3D and F3D10.5.1 Motion Parameter Coding

10.5.2 2D Shape Parameter Coding10.5.3 Coding of Component Separation10.5.4 Flexible Shape Parameter Coding10.5.5 Color Parameters

10.5.6 Control of Parameter Coding10.6 Experimental Results

10.7 ConclusionsReferences

11 Rate-Distortion Techniques in Image and Video Coding

Aggelos K Katsaggelos and Gerry Melnikov

11.1 The Multimedia Transmission Problem11.2 The Operational Rate-Distortion Function11.3 Problem Formulation

11.4 Mathematical Tools in RD Optimization11.4.1 Lagrangian Optimization11.4.2 Dynamic Programming11.5 Applications of RD Methods11.5.1 QT-Based Motion Estimation and Motion-Compensated Interpolation11.5.2 QT-Based Video Encoding

11.5.3 Hybrid Fractal/DCT Image Compression11.5.4 Shape Coding

11.6 ConclusionsReferences

12 Transform Domain Techniques for Multimedia Image and Video Coding

S Suthaharan, S.W Kim, H.R Wu, and K.R Rao

12.1 Coding Artifacts Reduction12.1.1 Introduction12.1.2 Methodology12.1.3 Experimental Results12.1.4 More Comparison12.2 Image and Edge Detail Detection12.2.1 Introduction

12.2.2 Methodology12.2.3 Experimental Results12.3 Summary

References

13 Video Modeling and Retrieval

Yi Zhang and Tat-Seng Chua

13.1 Introduction13.2 Modeling and Representation of Video: Segmentation vs

Stratification13.2.1 Practical Considerations13.3 Design of a Video Retrieval System13.3.1 Video Segmentation13.3.2 Logging of Shots13.3.3 Modeling the Context between Video Shots13.4 Retrieval and Virtual Editing of Video

13.4.1 Video Shot Retrieval13.4.2 Scene Association Retrieval13.4.3 Virtual Editing

13.5 Implementation13.6 Testing and Results13.7 ConclusionReferences

14 Image Retrieval in Frequency Domain Using DCT Coefficient Histograms

Jose A Lay and Ling Guan

14.1 Introduction14.1.1 Multimedia Data Compression14.1.2 Multimedia Data Retrieval14.1.3 About This Chapter14.2 The DCT Coefficient Domain14.2.1 A Matrix Description of the DCT14.2.2 The DCT Coefficients in JPEG and MPEG Media14.2.3 Energy Histograms of the DCT Coefficients14.3 Frequency Domain Image/Video Retrieval Using DCT Coefficients14.3.1 Content-Based Retrieval Model

14.3.2 Content-Based Search Processing Model14.3.3 Perceiving the MPEG-7 Search Engine14.3.4 Image Manipulation in the DCT Domain14.3.5 The Energy Histogram Features

14.3.6 Proximity Evaluation14.3.7 Experimental Results14.4 Conclusions

References

15 Rapid Similarity Retrieval from Image and Video

Kim Shearer, Svetha Venkatesh, and Horst Bunke

15.1 Introduction15.1.1 Definitions15.2 Image Indexing and Retrieval15.3 Encoding Video Indices15.4 Decision Tree Algorithms15.4.1 Decision Tree-Based LCSG Algorithm15.5 Decomposition Network Algorithm

15.5.1 Decomposition-Based LCSG Algorithm15.6 Results of Tests Over a Video Database

15.6.1 Decomposition Network Algorithm15.6.2 Inexact Decomposition Algorithm15.6.3 Decision Tree

15.6.4 Results of the LCSG Algorithms15.7 Conclusion

References

16 Video Transcoding

Tzong-Der Wu, Jenq-Neng Hwang, and Ming-Ting Sun

16.1 Introduction16.2 Pixel-Domain Transcoders16.2.1 Introduction16.2.2 Cascaded Video Transcoder16.2.3 Removal of Frame Buffer and Motion Compensation Modules16.2.4 Removal of IDCT Module

16.3 DCT Domain Transcoder16.3.1 Introduction16.3.2 Architecture of DCT Domain Transcoder16.3.3 Full-Pixel Interpolation

16.3.4 Half-Pixel Interpolation16.4 Frame-Skipping in Video Transcoding16.4.1 Introduction

16.4.2 Interpolation of Motion Vectors16.4.3 Search Range Adjustment16.4.4 Dynamic Frame-Skipping16.4.5 Simulation and Discussion16.5 Multipoint Video Bridging16.5.1 Introduction16.5.2 Video Characteristics in Multipoint Video Conferencing16.5.3 Results of Using the Coded Domain and Transcoding Approaches16.6 Summary

References

17 Multimedia Distance Learning

Sachin G Deshpande, Jenq-Neng Hwang, and Ming-Ting Sun

17.1 Introduction17.2 Interactive Virtual Classroom Distance Learning Environment17.2.1 Handling the Electronic Slide Presentation

17.2.2 Handling Handwritten Text17.3 Multimedia Features for On-Demand Distance Learning Environment17.3.1 Hypervideo Editor Tool

17.3.2 Automating the Multimedia Features Creation for On-Demand System17.4 Issues in the Development of Multimedia Distance Learning

17.4.1 Error Recovery, Synchronization, and Delay Handling17.4.2 Fast Encoding and Rate Control

17.4.3 Multicasting17.4.4 Human Factors17.5 Summary and ConclusionReferences

18 A New Watermarking Technique for Multimedia Protection

Chun-Shien Lu, Shih-Kun Huang, Chwen-Jye Sze, and Hong-Yuan Mark Liao

18.1 Introduction18.1.1 Watermarking18.1.2 Overview18.2 Human Visual System-Based Modulation18.3 Proposed Watermarking Algorithms18.3.1 Watermark Structures18.3.2 The Hiding Process18.3.3 Semipublic Authentication18.4 Watermark Detection/Extraction18.4.1 Gray-Scale Watermark Extraction18.4.2 Binary Watermark Extraction18.4.3 Dealing with Attacks Including Geometric Distortion18.5 Analysis of Attacks Designed to Defeat HVS-Based Watermarking18.6 Experimental Results

18.6.1 Results of Hiding a Gray-Scale Watermark18.6.2 Results of Hiding a Binary Watermark18.7 Conclusion

References

19 Telemedicine: A Multimedia Communication Perspective

Chang Wen Chen and Li Fan

19.1 Introduction19.2 Telemedicine: Need for Multimedia Communication19.3 Telemedicine over Various Multimedia Communication Links19.3.1 Telemedicine via ISDN

19.3.2 Medical Image Transmission via ATM19.3.3 Telemedicine via the Internet

19.3.4 Telemedicine via Mobile Wireless Communication19.4 Conclusion

References

Multimedia is one of the most important aspects of the information era Although there arebooks dealing with various aspects of multimedia, a book comprehensively covering system,processing, and application aspects of image and video data in a multimedia environment isurgently needed Contributed by experts in the field, this book serves this purpose

Our goal is to provide in a single volume an introduction to a variety of topics in image andvideo processing for multimedia An edited compilation is an ideal format for treating a broadspectrum of topics because it provides the opportunity for each topic to be written by an expert

in that field

The topic of the book is processing images and videos in a multimedia environment It coversthe following subjects arranged in two parts: (1) standards and fundamentals: standards, mul-timedia architecture for image processing, multimedia-related image processing techniques,and intelligent multimedia processing; (2) methodologies, techniques, and applications: im-age and video coding, image and video storage and retrieval, digital video transmission, videoconferencing, watermarking, distance education, video on demand, and telemedicine.The book begins with the existing standards for multimedia, discussing their impacts tomultimedia image and video processing, and pointing out possible directions for new standards.The design of multimedia architectures is based on the standards It deals with the wayvisual data is being processed and transmitted at a more practical level Current and newarchitectures, and their pros and cons, are presented and discussed in Chapters 2 to 4.Chapters 5 to 8 focus on conventional and intelligent image processing techniques relevant tomultimedia, including preprocessing, segmentation, and feature extraction techniques utilized

in coding, storage, and retrieval and transmission, media fusion, and graphical interface.Compression and coding of video and images are among the focusing issues in multimedia.New developments in transform- and motion-based algorithms in the compressed domain,content- and object-based algorithms, and rate–distortion-based encoding are presented inChapters 9 to 12

Chapters 13 to 15 tackle content-based image and video retrieval They cover video modelingand retrieval, retrieval in the transform domain, indexing, parsing, and real-time aspects ofretrieval

The last chapters of the book (Chapters 16 to 19) present new results in multimedia plication areas, including transcoding for multipoint video conferencing, distance education,watermarking techniques for multimedia processing, and telemedicine

ap-Each chapter has been organized so that it can be covered in 1 to 2 weeks when this book isused as a principal reference or text in a senior or graduate course at a university

It is generally assumed that the reader has prior exposure to the fundamentals of image andvideo processing The chapters have been written with an emphasis on a tutorial presentation

so that the reader interested in pursuing a particular topic further will be able to obtain a solidintroduction to the topic through the appropriate chapter in this book While the topics coveredare related, each chapter can be read and used independently of the others

This book is primarily a result of the collective efforts of the chapter authors We arevery grateful for their enthusiastic support, timely response, and willingness to incorporatesuggestions from us, from other contributing authors, and from a number of our colleagueswho served as reviewers.

Ling Guan Sun-Yuan Kung Jan Larsen