opencv 3 computer vision application robert laganiere7734

OpenCV 3 Computer Vision Application Programming Cookbook - Third Edition... Table of ContentsOpenCV 3 Computer Vision Application Programming Cookbook Third Edition -Credits About the A

OpenCV 3 Computer Vision Application Programming Cookbook - Third Edition

Table of Contents

OpenCV 3 Computer Vision Application Programming Cookbook Third Edition

-Credits

About the Author

About the Reviewer

www.PacktPub.com

Why subscribe?

Customer Feedback

Preface

What this book covers

What you need for this book

Who this book is for

Downloading the example code

Downloading the color images of this book

The input and output arrays

Manipulating small matrices

Getting ready

How to do it

How it works

There's more

Other color reduction formulas

Having input and output arguments

Efficient scanning of continuous imagesLow-level pointer arithmetic

Overloaded image operators

Splitting the image channels

Remapping an image

How to do it

How it works

The floodFill function

Functor or function object

The OpenCV base class for algorithms

Applying look-up tables to modify the image's appearanceHow to do it

Describing and matching local intensity patterns

Refining the fundamental matrix

Refining the matches

Computing a homography between two images

Processing a sequence of images

Using a frame processor class

14 Learning from Examples

OpenCV 3 Computer Vision Application Programming Cookbook - Third Edition

OpenCV 3 Computer Vision

Application Programming

Cookbook - Third Edition

Copyright © 2017 Packt Publishing

All rights reserved No part of this book may be reproduced, stored in aretrieval system, or transmitted in any form or by any means, withoutthe prior written permission of the publisher, except in the case of briefquotations embedded in critical articles or reviews

Every effort has been made in the preparation of this book to ensure theaccuracy of the information presented However, the information

contained in this book is sold without warranty, either express or

implied Neither the author, nor Packt Publishing, and its dealers anddistributors will be held liable for any damages caused or alleged to becaused directly or indirectly by this book

Packt Publishing has endeavored to provide trademark information

about all of the companies and products mentioned in this book by theappropriate use of capitals However, Packt Publishing cannot guaranteethe accuracy of this information

First published: May 2011

Second edition: August 2014

Third edition: February 2017

Production reference: 1070217

Published by Packt Publishing Ltd

Livery Place

Tejal Daruwale Soni

Content Development Editor

Nikhil Borkar

Graphics

Kirk D'Penha Jason Monteiro

Technical Editor

Subhalaxmi Nadar

Production Coordinator

Shantanu Zagade

About the Author

Robert Laganiere is a professor at the School of Electrical Engineering

and Computer Science of the University of Ottawa, Canada He is also afaculty member of the VIVA research lab and is the co-author of severalscientific publications and patents in content-based video analysis,

visual surveillance, driver-assistance, object detection, and tracking.Robert authored the OpenCV2 Computer Vision Application

Programming Cookbook in 2011 and co-authored Object Oriented

Software Development published by McGraw Hill in 2001 He

co-founded Visual Cortek in 2006, an Ottawa-based video analytics startupthat was later acquired by http://iwatchlife.com/ in 2009 He is also aconsultant in computer vision and has assumed the role of Chief

Scientist in a number of startups companies such as Cognivue Corp,iWatchlife, and Tempo Analytics Robert has a Bachelor of ElectricalEngineering degree from Ecole Polytechnique in Montreal (1987) andMSc and PhD degrees from INRS-Telecommunications, Montreal

(1996) Visit the author’s website at http://www.laganiere.name/

I wish to thank all my students at the VIVA lab; I learn so much

from them.

About the Reviewer

Luca Del Tongo is a computer engineer with a strong passion for

algorithms, computer vision, and image processing techniques He's thecoauthor of a free e-book called Data Structures and Algorithms (DSA)with over 100k downloads so far and has published several image

processing tutorials on his YouTube channel using Emgu CV During hismaster's thesis, he developed an image forensic algorithm published in ascientific paper called Copy Move forgery detection and localization bymeans of robust clustering with J-Linkage Currently, Luca works as asoftware engineer in the ophthalmology field developing corneal

topography, processing algorithms, IOL calculation, and computerizedchart projector He loves to play sport and follow MOOC courses in hisspare time

You can contact him through his blog

at http://blogs.ugidotnet.org/wetblog

For support files and downloads related to your book, please

visit www.PacktPub.com

Did you know that Packt offers eBook versions of every book

published, with PDF and ePub files available? You can upgrade to theeBook version at www.PacktPub.com and as a print book customer, youare entitled to a discount on the eBook copy Get in touch with us atservice@packtpub.com for more details

At www.PacktPub.com, you can also read a collection of free technicalarticles, sign up for a range of free newsletters and receive exclusivediscounts and offers on Packt books and eBooks

https://www.packtpub.com/mapt

Get the most in-demand software skills with Mapt Mapt gives you fullaccess to all Packt books and video courses, as well as industry-leadingtools to help you plan your personal development and advance yourcareer

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Customer Feedback

Thank you for purchasing this Packt book We take our commitment toimproving our content and products to meet your needs seriously—that'swhy your feedback is so valuable Whatever your feelings about yourpurchase, please consider leaving a review on this book's Amazon page.Not only will this help us, more importantly it will also help others in thecommunity to make an informed decision about the resources that theyinvest in to learn

You can also review for us on a regular basis by joining our reviewers'

club If you're interested in joining, or would like to learn more

about the benefits we offer, please contact us:

customerreviews@packtpub.com

advance research in vision and promote the development of rich based, CPU-intensive applications After a series of beta releases,

vision-version 1.0 was launched in 2006 A second major release occurred in

2009 with the launch of OpenCV 2 that proposed important changes,especially the new C++ interface, which we use in this book In 2012,OpenCV reshaped itself as a non-profit foundation (http://opencv.org/)relying on crowdfunding for its future development OpenCV3 was

introduced in 2013; changes were made mainly to improve the usability

of library Its structure has been revised to remove the unnecessary

dependencies, large modules have been split into smaller ones and the

API has been refined This book is the third edition of the OpenCV

Computer Vision Application Programming Cookbook and the first one

that covers OpenCV version 3 All the programming recipes of the

previous editions have been reviewed and updated We also have addednew content and new chapters to provide readers with even better

coverage of the essential functionalities of the library This book coversmany of the library’s features and explains how to use them to

accomplish specific tasks Our objective is not to provide detailed

coverage of every option offered by the OpenCV functions and classesbut rather to give you the elements you need to build your applicationsfrom the ground up We also explore, in this book, fundamental concepts

in image analysis and we describe some of the important algorithms incomputer vision This book is an opportunity for you to get introduced

to the world of image and video analysis But this is just the beginning.The good news is that OpenCV continues to evolve and expand Justconsult the OpenCV online documentation at http://opencv.org/ to stayupdated about what the library can do for you You can also visit theauthor’s website at http://www.laganiere.name/ for updated informationabout this cookbook

What this book covers

Chapter 1 , Playing with Images, introduces the OpenCV library and

shows you how to build simple applications that can read and displayimages It also introduces the basic OpenCV data structures

Chapter 2 , Manipulating Pixels, explains how an image can be read It

describes different methods for scanning an image in order to perform

an operation on each of its pixels

Chapter 3 , Processing the Colors of an Image, consists of recipes

presenting various object-oriented design patterns that can help you tobuild better computer vision applications It also discusses the concept

of colors in images

Chapter 4 , Counting the Pixels with Histograms, shows you how to

compute image histograms and how they can be used to modify an

image Different applications based on histograms are presented thatachieve image segmentation, object detection, and image retrieval

Chapter 5 , Transforming Images with Morphological Operations,

explores the concept of mathematical morphology It presents differentoperators and how they can be used to detect edges, corners, and

segments in images

Chapter 6 , Filtering the Images, teaches you the principle of frequency

analysis and image filtering It shows how low-pass and high-pass filterscan be applied to images and presents the concept of derivative

operators

Chapter 7 , Extracting Lines, Contours, and Components, focuses on the

detection of geometric image features It explains how to extract

contours, lines and connected components in an image

Chapter 8 , Detecting Interest Points, describes various feature point

detector in images

Chapter 9 , Describing and Matching Interest Points, explains how

descriptors of interest points can be computed and used to match pointsbetween images

Chapter 10 , Estimating Projective Relations in Images, explores the

projective relations that exist between two images in the same scene Italso describes how to detect specific targets in an image

Chapter 11 , Reconstructing 3D scenes, allows you to reconstruct the 3D

elements of a scene from multiple images and recover the camera pose

It also includes a description of the camera calibration process

Chapter 12 , Processing Video Sequences, provide a framework to read

and write a video sequence and to process its frames It shows you alsohow it is possible to extract the foreground objects moving in front of acamera

Chapter 13 , Tracking Visual Motion, addresses the visual tracking

problem It shows you how to compute the apparent motion in videos Italso explains how to track moving objects in an image sequence

Chapter 14 , Learning from Examples, introduces basic concepts in

machine learning It shows how object classifiers can be built from

image samples

What you need for this book

This cookbook is based on the C++ API of the OpenCV library It istherefore assumed that you have some experience with the C++

language In order to run the examples presented in the recipes and

experiment with them, you need a good C++ development environment.Microsoft Visual Studio and Qt are two popular choices

Who this book is for

This cookbook is appropriate for novice C++ programmers who want tolearn how to use the OpenCV library to build computer vision

applications It is also suitable for professional software developers whowants to be introduced to the concepts of computer vision programming

It can be used as a companion book in university-level computer visioncourses It constitutes an excellent reference for graduate students andresearchers in image processing and computer vision

In this book, you will find several headings that appear frequently

(Getting ready, How to do it, How it works, There's more, and See also)

To give clear instructions on how to complete a recipe, we use thesesections as follows:

This section usually consists of a detailed explanation of what happened

in the previous section

There's more…

This section consists of additional information about the recipe in order

to make the reader more knowledgeable about the recipe

See also

This section provides helpful links to other useful information for therecipe

In this book, you will find a number of text styles that distinguish

between different kinds of information Here are some examples of

these styles and an explanation of their meaning

Code words in text, database table names, folder names, filenames, fileextensions, pathnames, dummy URLs, user input, and Twitter handlesare shown as follows: "We can include other contexts through the use ofthe include directive."

A block of code is set as follows:

// Compute Laplacian using LaplacianZC class

LaplacianZC laplacian;

laplacian.setAperture(7); // 7x7 laplacian

cv::Mat flap= laplacian.computeLaplacian(image);

laplace= laplacian.getLaplacianImage();

When we wish to draw your attention to a particular part of a code

block, the relevant lines or items are set in bold:

// Compute Laplacian using LaplacianZC class

LaplacianZC laplacian;

laplacian.setAperture(7); // 7x7 laplacian

cv::Mat flap= laplacian.computeLaplacian(image);

laplace= laplacian.getLaplacianImage();

New terms and important words are shown in bold Words that you

see on the screen, for example, in menus or dialog boxes, appear in the

text like this: "Clicking the Next button moves you to the next screen."

Reader feedback

Feedback from our readers is always welcome Let us know what youthink about this book-what you liked or disliked Reader feedback isimportant for us as it helps us develop titles that you will really get themost out of

To send us general feedback, simply e-mail feedback@packtpub.com,and mention the book's title in the subject of your message

If there is a topic that you have expertise in and you are interested ineither writing or contributing to a book, see our author guide

at www.packtpub.com/authors

Customer support

Now that you are the proud owner of a Packt book, we have a number

of things to help you to get the most from your purchase

Downloading the example code

You can download the example code files for this book from your

account at http://www.packtpub.com If you purchased this book

elsewhere, you can visit http://www.packtpub.com/support and register

to have the files e-mailed directly to you

You can download the code files by following these steps:

1 Log in or register to our website using your e-mail address and

password

2 Hover the mouse pointer on the SUPPORT tab at the top.

3 Click on Code Downloads & Errata.

4 Enter the name of the book in the Search box.

5 Select the book for which you're looking to download the code files

6 Choose from the drop-down menu where you purchased this bookfrom

7 Click on Code Download.

You can also download the code files by clicking on the Code Files

button on the book's webpage at the Packt Publishing website This page

can be accessed by entering the book's name in the Search box Please

note that you need to be logged in to your Packt account

Once the file is downloaded, please make sure that you unzip or extractthe folder using the latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac

7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at

https://github.com/PacktPublishing/OpenCV3-Computer-Vision-Application-Programming-Cookbook-Third-Edition We also have other

code bundles from our rich catalog of books and videos available at

https://github.com/PacktPublishing/ Check them out!

The source code files of the examples presented in this cookbook are

also hosted in the author's Github repository You can visit the author's

repository at https://github.com/laganiere to obtain the latest version of

the code

Downloading the color images of this book

We also provide you with a PDF file that has color images of the

screenshots/diagrams used in this book The color images will help you

better understand the changes in the output You can download this file

from https://www.packtpub.com/sites/default/files/downloads/OpenCV3ComputerVisionApplicationProgrammingCookbookThirdEdition_ColorImages.pdf

Errata

Although we have taken every care to ensure the accuracy of our

content, mistakes do happen If you find a mistake in one of our

books-maybe a mistake in the text or the code-we would be grateful if you

could report this to us By doing so, you can save other readers from

frustration and help us improve subsequent versions of this book If you

find any errata, please report them by visiting

http://www.packtpub.com/submit-errata, selecting your book, clicking

on the Errata Submission Form link, and entering the details of your

errata Once your errata are verified, your submission will be accepted

and the errata will be uploaded to our website or added to any list of

existing errata under the Errata section of that title

To view the previously submitted errata, go to

https://www.packtpub.com/books/content/support and enter the name of

the book in the search field The required information will appear under

the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problemacross all media At Packt, we take the protection of our copyright andlicenses very seriously If you come across any illegal copies of ourworks in any form on the Internet, please provide us with the locationaddress or website name immediately so that we can pursue a remedy.

Please contact us at copyright@packtpub.com with a link to the

suspected pirated material

We appreciate your help in protecting our authors and our ability tobring you valuable content

Questions

If you have a problem with any aspect of this book, you can contact us

at questions@packtpub.com, and we will do our best to address theproblem

Chapter 1 Playing with Images

In this chapter, we will get you started with the OpenCV library You

will learn how to perform the following tasks:

Installing the OpenCV library

Loading, displaying, and saving images

Exploring the cv::Mat data structure

Defining regions of interest

Introduction

This chapter will teach you the basic elements of OpenCV and will showyou how to accomplish the most fundamental image processing tasks:reading, displaying, and saving images However, before you start withOpenCV, you need to install the library This is a simple process that isexplained in the first recipe of this chapter

All your computer vision applications will involve the processing of

images This is why OpenCV offers you a data structure to handle

images and matrices It is a powerful data structure with many usefulattributes and methods It also incorporates an advanced memory

management model that greatly facilitates the development of

applications The last two recipes of this chapter will teach you how touse this important data structure of OpenCV

Installing the OpenCV library

OpenCV is an open source library for developing computer vision

applications that can run on multiple platforms, such as Windows,

Linux, Mac, Android, and iOS It can be used in both academic andcommercial applications under a BSD license that allows you to freelyuse, distribute, and adapt it This recipe will show you how to install thelibrary on your machine

Getting ready

When you visit the OpenCV official website at http://opencv.org/ , youwill find the latest release of the library, the online documentation

describing the Application Programming Interface (API), and many

other useful resources on OpenCV

How to do it

From the OpenCV website, find the latest available downloads and

select the one that corresponds to the platform of your choice

(Windows, Linux/Mac, or iOS) Once the OpenCV package is

downloaded, run the WinZip self-extractor and select the location ofyour choice An opencv directory will be created; it is a good idea torename it in a way that will show which version you are using (for

example, in Windows, your final directory could be C:\opencv-3.2).This directory will contain a collection of files and directories that

constitute the library Notably, you will find the sources directory thatwill contain all the source files (yes, it is open source!)

In order to complete the installation of the library and have it ready foruse, you need to take an important step: generate the binary files of thelibrary for the environment of your choice This is indeed the point

where you have to make a decision on the target platform you wish touse to create your OpenCV applications Which operating system doyou prefer to use? Which compiler should you select? Which version?32-bit or 64-bit? As you can see, there are many possible options, and

this is why you have to build the library that fits your needs.

The Integrated Development Environment (IDE) you will use in your

project development will also guide you to make these choices Notethat the library package also comes with precompiled binaries that youcan directly use if they correspond to your situation (check the build

directory adjacent to the sources directory) If one of the precompiledbinaries satisfies your requirements, then you are ready to go

One important remark, however Since version 3, OpenCV has been splitinto two major components The first one is the main OpenCV sourcerepository that includes the mature algorithms This is the one you havedownloaded A separate contribution repository also exists, and it

contains the new computer vision algorithm, recently added by the

OpenCV contributors If your plan is to use only the core functions ofOpenCV, you do not need the contrib package But if you want to playwith the latest state-of-the-art algorithms, then there is a good chancethat you will need this extra module As a matter of fact, this cookbookwill show you how to use several of these advanced algorithms Youtherefore need the contrib modules to follow the recipes of this book

So you have to go to https://github.com/opencv/opencv_contrib anddownload OpenCV's extra modules (download the ZIP file) You canunzip the extra modules into the directory of your choice; these modulesshould be found at opencv_contrib-master/modules For simplicity, youcan rename this directory as contrib and copy it directly inside the

sources directory of the main package Note that you can also pick theextra modules of your choice and only save them; however, you willprobably find it easier, at this point, to simply keep everything

You are now ready to proceed with the installation To build the

OpenCV binaries, it is highly suggested that you use the CMake tool,

available at http://cmake.org CMake is another open source softwaretool designed to control the compilation process of a software systemusing platform-independent configuration files It generates the required

makefile or solution files needed for compiling a software library inyour environment Therefore, you have to download and install CMake

Also see the There's more section of this recipe for an additional

software package, the Visualization Toolkit (VTK), that you may want

to install before compiling the library

You can run cmake using a command-line interface, but it is easier to use

CMake with its graphical interface (cmake-gui) In the latter case, all you

need to do is specify the folder containing the OpenCV library source

and the one that will contain the binaries Now click on Configure and

select the compiler of your choice:

Once this initial configuration is completed, CMake will provide youwith a number of configuration options You have to decide, for

example, whether you want to have the documentation installed or

whether you wish to have some additional libraries installed Unless you

know what you are doing, it is probably better to leave the default

options as they are However, since we want to include the extra

modules, we have to specify the directory where they can be found:

Once the extra module path is specified, click on Configure again You are now ready to generate the project files by clicking on the Generate

button These files will allow you to compile the library This is the laststep of the installation process, which will make the library ready to beused in your development environment For example, if you select MSVisual Studio, then all you need to do is open the top-level solution filethat CMake has created for you (the OpenCV.sln file) You then select

the INSTALL project (under CMakeTargets) and issue the Build

command (use right-click)

To get both a Release and Debug build, you will have to repeat the

compilation process twice, one for each configuration If everythinggoes well, you will have an install directory (under build) created.This directory will contain all the binary files of the OpenCV library to

be linked with your application as well as the dynamic library files thatyour executables have to call at runtime Make sure you set your

system's PATH environment variable (from Control Panel) such that

your operating system would be able to find the .dll files when you runyour applications (for example, C:\opencv-3.2\build

\install\x64\vc14\bin) You should also define the environment

variable, OPENCV_DIR pointing to the INSTALL directory This way, CMakewill be able to find the library when configuring future projects

In Linux environments, you can use Cmake to generate the required

Makefiles; you then complete the installation by executing a sudo make install command Alternatively, you could also use the packaging tool

apt-get which can automatically perform a complete installation of thelibrary For Mac OS, you should use the Homebrew package manager.Once installed, you just have to type brew install opencv3 with- contrib in order to have the complete library installed (run brew info opencv3 to view all possible options)

How it works

OpenCV is a library that is in constant evolution With version 3, thelibrary continues to expand offering a lot of new functionalities withenhanced performances The move to having a full C++ API, which wasinitiated in version 2, is now almost complete, and more uniform

interfaces have been implemented One of the major changes introduced

in this new version is the restructuring of the modules of the library inorder to facilitate its distribution In particular, a separate repository

containing the most recent algorithms has been created This contrib

repository also contains non-free algorithms that are subject to specificlicenses The idea is for OpenCV to be able to offer state-of-the-art

functionalities that developers and researchers want to share while stillbeing able to offer a very stable and well-maintained core API The

main modules are therefore the ones you get when you download thelibrary at http://opencv.org/ The extra modules must be downloadeddirectly from the development repository hosted on GitHub (

https://github.com/opencv/ ) Since these extra modules are in constantdevelopment, you should expect more frequent changes to the

algorithms they contain

The OpenCV library is divided into several modules For example, the