Digital Signal Processing Handbook P58

Reeves Auburn University 58.1 Image Processing Software General Image Utilities • Specialized Image Utilities• Pro-gramming/Analysis Environments 58.2 Image Databases Images by Form Imag

Reeves, S.J “A Survey of Image Processing Software and Image Databases”

Digital Signal Processing Handbook

Ed Vijay K Madisetti and Douglas B Williams

Boca Raton: CRC Press LLC, 1999

A Survey of Image Processing Software and Image Databases

Stanley J Reeves

Auburn University

58.1 Image Processing Software General Image Utilities • Specialized Image Utilities•

Pro-gramming/Analysis Environments 58.2 Image Databases

Images by Form

Image processing has moved into the mainstream, not only of the engineering world, but of society

in general Personal computers are now capable of handling large graphics and images with ease, and fast networks and modems transfer images in a fraction of the time required just a few years ago Image manipulation software is a common item on PCs, and CD-ROMs filled with images and multimedia databases are standard fare in the realm of electronic publishing Furthermore, the development of areas such as data compression, neural networks and pattern recognition, computer vision, and multimedia systems have all contributed to the use of and interest in image processing Likewise, the growth of image processing as an engineering discipline has fueled interest in these other areas As a result of this symbiotic growth, image processing has increasingly become a standard tool

in the repertoire of the engineer

Because of the popularity of image processing, a large array of tools has emerged for accomplishing various image processing tasks In addition, a variety of image databases has been created to address the needs of various specialty areas In this article, we will survey some of the tools available for accomplishing basic image processing tasks and indicate where they may be obtained Furthermore,

we will describe and provide pointers to some of the most generally useful images and image databases The goal is to identify a basic collection of images and software that will be of use to the nonspecialist

It should also be of use to the specialist who needs a general tool in an area outside his or her specialty

58.1 Image Processing Software

Image processing has become such a broad area that it is sometimes difficult to distinguish what might

be considered an image processing package from other software systems The boundaries among the areas of computer graphics, data visualization, and image processing have become blurred Furthermore, to discuss or even to list all the image processing software available would require many pages and would not be particularly useful to the nonspecialist Therefore, we emphasize a representative set of image processing software packages that embody core capabilities in scientific image processing applications Core capabilities, in our view, include the following:

• Image utilities: These include display, manipulation, and file conversion Images come

in such a variety of formats that a package for converting images from one format to another is essential Furthermore, basic display and manipulation (cropping, rotating, etc.) are essential for almost any image processing task The ability to edit images using cut-and-paste, draw, and annotate operations is also useful in many cases

• Image filtering and transformation: These are necessary capabilities for most scientific

applications of image processing Convolution, median filtering, FFTs, morphological operations, scaling, and other image functions form the core of many scientific image processing algorithms

• Image compression: Anyone who works with images long enough will learn that they

require a large amount of storage space A number of standard image compression utilities are available for storing images in compressed form and for retrieving compressed images from image databases

• Image analysis: Scientific image processing applications often have the goal of deriving

information from an image Simple image analysis tools such as edge detection and segmentation are powerful methods for gleaning important visual information

• Programming and data analysis environment: While many image processing packages

have a wide variety of functions, a whole new level of utility and flexibility arises when the image processing functions are built around a programming and/or data analysis en-vironment Programming environments allow for tailoring image processing techniques

to the specific task, developing new algorithms, and interfacing image processing tasks with other scientific data analysis and numerical computational techniques

Other capabilities include higher-level object recognition and other computer vision tasks, visual-ization and rendering techniques, computed imaging such as medical image reconstruction, and morphing and other special effects of the digital darkroom and the film industry These areas require highly specialized software and/or very specialized skills to apply the methods and are not likely to

be part of the image processing world of the nonspecialist

The packages to be discussed here encompass as a group all of the core image processing capabilities mentioned above Because these packages offer such a wide variety and mix of functions, they defy simple categorization We have chosen to group the packages into three categories: general image utilities, specialized utilities, and programming/analysis environments Keep in mind, however, that the distinctions among these groups is blurry at best We have chosen to emphasize packages that are freely distributable and available on the Internet because these can be obtained and used with a minimum of expense and hassle

58.1.1 General Image Utilities

netpbm

pbmplus is a set of tools that allows the user to convert to and from a large number of common image formats The package has its own intermediate formats so that the conversion routines can

be written to convert to or from one of these formats The user can then convert to and from any combination of formats by going through one of the intermediate formats Functions are also provided to convert from different color resolutions, such as from color to grayscale Several other functions do basic image manipulation such as cropping, rotating, and smoothing The source is available from

ftp://ftp.wustl.edu/graphics/graphics/packages/NetPBM/

xv is an X11 utility that combines several important image handling functions It can display images in a wide variety of display formats, including binary, 8-bit, and 24-bit It allows the user to manipulate the colormap both in RGB and HSV space It crops, resizes, smooths, rotates, detects edges, and produces other special effects In addition, it reads and writes a large variety of image formats, so it can serve as a format conversion utility Until recently, xv has been freely distributable The latest version, however, is shareware and requires a small fee to become a registered user The source is available from

http://www.trilon.com/xv

NCSA Image

NCSA Image is available in versions for the Mac, DOS, and Unix (X11) The Unix version is called ximage ximage allows the user to display color images It can also display the actual data in the form of a spreadsheet A number of other display options are available Like xv, it allows for manipulation of the colormap in a variety of ways In addition, the user may display multiple images

as an animated sequence, either from disk or server memory The functionality of NCSA Image is augmented by other programs available from NCSA, including DataSlice for visualization tasks and Reformat for converting image formats The source is available from NCSA by ftp at

ftp://ftp.ncsa.uiuc.edu/Visualization/Image/

ImageMagick

ImageMagick is an X11 package for display and interactive image manipulation It reads and writes a large number of standard formats, does standard operations such as cropping and rotating

as well as more specialized editing operations such as cutting, pasting, color filling, annotating, and drawing Separate utilities are provided for grabbing images from a display, for converting, combining, resizing, blurring, adding borders, and doing many other operations The source is available by ftp from

ftp://ftp.x.org/contrib/applications/ImageMagick/

NIH Image

NIH Image is available only in a Macintosh version However, the popularity of NIH Image among Mac users and the breadth of features justify inclusion of the package in this survey It reads/writes a small number of image formats, acquires images using compatible frame grabbers, and displays It allows image manipulation such as flipping, rotating, and resizing; and editing such

as drawing and annotating It has a number of built-in enhancement and filtering functions: contrast enhancement, smoothing, sharpening, median filtering, and convolution It supports a number of analysis operations such as edge detection and measurement of area, mean, centroid, and perimeter

of user-defined regions of interest It also performs automated particle analysis In addition, the user can animate a set of images NIH Image has a Pascal-like macro capability and the ability to add precompiled plug-in modules The source is available from NIH by ftp at

ftp://zippy.nimh.nih.gov/pub/nih-image/

LaboImage

LaboImage is an X11 package for mouse- and menu-driven interactive image processing It reads/writes a special format as well as Sun raster format and displays grayscale and RGB and provides dithering Basic filtering operations are possible, as well as enhancement tasks such as background subtraction and histogram equalization It computes various measures such as histograms, image statistics, and image power Region outlining and object counting can be done as well Images can

be modified interactively at the pixel level, and an expert system is available for region segmentation LaboImage has a macro capability for combining operations LaboImage can be obtained from http://cuiwww.unige.ch/ftp/sgaico/research/geneve/vision/labo.html

Paint Shop Pro

Paint Shop Pro is a Windows-based package for creating, displaying, and manipulating images

It has a large number of image editing features, including painting, photo retouching, and color en-hancement It reads and writes a large number of formats It includes several standard image process-ing filters and geometrical transformations It can be obtained from http://www.jasc.com/psp.html

It is shareware and costs $69

58.1.2 Specialized Image Utilities

Compression

JPEG is a standard for image compression developed by the Joint Photographic Experts Group Free, portable C code that implements JPEG compression and decompression has been developed by the Independent JPEG Group, a volunteer organization It is available from ftp://ftp.uu.net/graphics/jpeg The downloadable package contains source and documentation The code converts between JPEG and several other common image formats A lossless JPEG implementation can be obtained from ftp://ftp.cs.cornell.edu/pub/multimed/

A fractal image compression program is available from ftp://inls.ucsd.edu/pub/young-fractal/ The package contains source for both compression and decompression A number of other fractal compression programs are also available and can be found in the sci.fractal FAQ at

ftp://rtfm.mit.edu/pub/usenet/news.answers/sci/fractals-faq

JBIG is a standard for binary image compression developed by the Joint Binary Images Group A JBIG coder/decoder can be obtained from ftp://nic.funet.fi/pub/graphics/misc/test-images/

MPEG is a standard for video/audio compression developed by the Moving Pictures Experts Group

A set of MPEG tools is available from ftp://mm-ftp.cs.berkeley.edu/pub/multimedia/mpeg/ These tools allow for encoding, decoding (playing), and analyzing the MPEG data

H.261 and H.263 are standards for video compression for videophone applications An H.261 coder/decoder is available from ftp://havefun.stanford.edu/pub/p64/ An H.263 video coder/decoder

is available from http://www.fou.telenor.no/brukere/DVC/h263 software/

Computer Vision

Vista is an X11-based image processing environment specifically designed for computer vision applications It allows a variety of display and manipulation options It has a library that lets the user easily create applications with menus, mouse interaction, and display options Vista defines a very flexible data format that represents a variety of images, collections of images, or other objects It also has the ability to add new objects or new image attributes without changing existing software or data files It does edge detection and linking, optical flow estimation and camera calibration, and viewing

of images and edge vectors Vista includes routines for common image processing operations such

as convolution, FFTs, simple enhancement tasks, scaling, cropping, and rotating Vista is available from

http://www.cs.ubc.ca/nest/lci/vista/vista.html

58.1.3 Programming/Analysis Environments

Khoros

Khoros is a comprehensive software development and data analysis environment It allows the user to perform a large variety of image and signal processing and visualization tasks A graphical programming environment called Cantata allows the user to construct programs visually using a data flowgraph approach It has a user interface design tool with automatic code generation for writing customized applications Software objects (programs) are accessible from the command line, from within Cantata, and in libraries

A large set of standard numerical and statistical algorithms are available within Khoros Common image processing operations such as FFTs, convolution, median filtering, and morphological oper-ators are available In addition, a variety of image display and geometrical manipulation programs, animation, and colormap editing are included Khoros has a very general data model that allows for images of up to five dimensions

Khoros is free-access software — it is available for downloading free of charge but cannot be dis-tributed without a license It can be obtained from Khoral Research, Inc., at ftp://ftp.khoral.com/pub/ Note that the Khoros distribution is quite large and requires significant disk space

MATLAB

MATLAB is a general numerical analysis and visualization environment Matrices are the underlying data structure in MATLAB, and this structure lends itself well to image processing ap-plications All data in MATLAB is represented as double-precision, which makes the calculations more precise and interaction more convenient However, it may also mean that MATLAB uses more memory and processing time than necessary

A large number of numerical algorithms and visualization options are available with the standard package The Image Processing Toolbox provides a great deal of added functionality for image processing applications It reads/writes several of the most common image formats; does convolution, FFTs, median filtering, histogram equalization, morphological operations, two-dimensional filter design, general nonlinear filtering, colormap manipulation, and basic geometrical manipulation It also allows for a variety of display options, including surface warping and movies

MATLAB is an interactive environment, which makes interactive image processing and manipula-tion convenient One can also add funcmanipula-tionality by creating scripts or funcmanipula-tions that use MATLAB’s functions and other user-added functions Additionally, one can add functions that have been writ-ten in C or Fortran Conversely, C or Fortran programs can call MATLAB and MATLAB library functions MATLAB is commercial software More information on MATLAB and how to obtain it can be found through the homepage of The Mathworks, Inc., at http://www.mathworks.com/

PV-Wave

PV-Wave is a general graphical/visualization and numerical analysis environment It can handle images of arbitrary dimensionality — 2-D, 3-D, and so on The user can specify the data type of each data structure, which allows for flexibility but may be inconvenient for interactive work

PV-Wave contains a large collection of visualization and rendering options, including colormap manipulation, volume rendering, and animation In addition, the IMSL library is available through PV-Wave Basic image processing operations such as convolution, FFTs, median filtering, morpho-logical operations, and contrast enhancement are included

Like MATLAB, PV-Wave is an interactive environment One can create scripts or functions from the PV-Wave language to add functionality It can also call C or Fortran functions PV-Wave can

be invoked from within C or Fortran too PV-Wave is commercial software More information on PV-Wave can be found through the homepage of Visual Numerics, Inc., at http://www.vni.com/

58.2 Image Databases

A huge number of image databases and archives are available on the Internet now, and more are continually being added These databases serve various purposes For the practicing engineer, the primary value of an image database is for developing, testing, evaluating, or comparing image processing and manipulation algorithms Standard images provide a benchmark for comparing various algorithms Furthermore, standard test images can be selected so that their characteristics are particularly suited to demonstrating the strengths and weaknesses of particular types of image processing techniques In some areas of image processing no real standards exist, although de facto standards have arisen In the discussion that follows, we provide pointers to some standard images, some de facto standards, and a few other databases that might provide images of value to algorithm work in image processing We have deliberately steered away from images whose copyright is known

to prohibit use for research purposes However, some of the images in the list have certain copyright restrictions Be sure to check any auxiliary information provided with the images before assuming that they are public domain

The images listed are in a variety of formats and may require conversion using one of the packages discussed previously such as netpbm We list the databases according to two categories: (1) form and (2) content By form, we mean that the images are organized according to the form of the image

— color, stereo, sequence, etc By content, we mean that the images are grouped according to the image content — faces, fingerprints, etc

58.2.1 Images by Form

Binary Images

A set of standard CCITT fax test images has been made available for testing compression schemes These are binary images that have come from scanning actual documents They can be found at ftp://nic.funet.fi/pub/graphics/misc/test-images/ under ccitt[1-8].pbm.gz

Grayscale Images

A collection of grayscale images can be obtained from ftp://ipl.rpi.edu/pub/image/still/canon/gray/

A compilation of de facto standard images can be found at

http://www.sys.uea.ac.uk/Research/ResGroups/SIP/images ftp/index.html

Note that the Lena image is copyrighted and should not be used in publications

Color Images

A set of test images that were used by the JPEG committee in the development of the JPEG algorithm are available from ftp://ipl.rpi.edu/pub/image/still/jpeg/bgr/

These are 24-bit RGB images Other 24-bit color images can be found at

ftp://ipl.rpi.edu/pub/image/still/canon/bgr/

A set of miscellaneous images in JPEG and Kodak CD format can be found at

http://www.kodak.com/digitalImages/samples/samples.shtml

Image Sequences

Image sequences may be intended for study of computer vision applications or video coding A huge set of sequences for computer vision applications are archived at

http://www.ius.cs.cmu.edu/idb/

A set of sequences commonly used for video coding applications can be found at

ftp://ipl.rpi.edu/pub/image/sequence/

Stereo Image Pairs

Stereo image pairs are available from http://www.ius.cs.cmu.edu/idb/.

Texture Images

A large set of texture images can be found at

http://www-white.media.mit.edu/vismod/imagery/VisionTexture/vistex.html

These images include textures from various angles and under different lighting conditions

Face Images

The USENIX FACES database contains hundreds of face images in various formats The database

is archived at ftp://ftp.uu.net/published/usenix/faces/

Fingerprint Images

Fingerprint images can be obtained from ftp://sequoyah.ncsl.nist.gov/pub/databases/data/

Medical Images

A variety of medical images are available over the Internet An excellent collection of CT, MRI, and cryosection images of the human body has been made available by the National Library of Medicine’s The Visual Human Project Samples of these images can be acquired at

http://www.nlm.nih.gov/research/visible/visible human.html

A collection of over 3500 images that cover an entire human body is available via ftp and on tape by signing a license agreement

MRI and CT volume images are available from

ftp://omicron.cs.unc.edu/pub/projects/softlab.v/CHVRTD/

PET images and other modalities can be found in

gopher://gopher.austin.unimelb.edu.au/11/images/petimages

Astronomical Images

A collection of astronomical images can be found at

https//www.univ-rennesl.fr/ASTRO/astro.english.html

Hubble telescope imagery can be obtained from http://archive.stsci.edu/archive.html

Range Images

Range images are available from

http://www.eecs.wsu.edu/˜irl/3DDB/RID/, along with a list of other sources of range imagery, and also from http://marathon.csee.usf.edu/range/DataBase.html

The tools and databases discussed here should provide a convenient set of capabilities for the nonspecialist The capabilities that are readily available are not static, however Image processing will continue to become more and more mainstream, so we expect to see the development of image processing tools representing greater variety and sophistication The advent of the World Wide Web will also stimulate further development and publishing of image databases on the Internet Therefore, image processing capabilities will continue to grow and will be more readily available The items discussed here are only a small sample of what will be available as time goes on