(Đồ án hcmute) research and implement a model of mango softing machine with image processing applications

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING Ho Chi Minh City, Febuary, 2023 SKL 0 1 0 4 2 8 RESEARCH AN

MINISTRY OF EDUCATION AND TRAINING

HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION

FACULTY FOR HIGH QUALITY TRAINING

Ho Chi Minh City, Febuary, 2023

SKL 0 1 0 4 2 8

RESEARCH AND IMPLEMENT A MODEL OF MANGO SORTING MACHINE WITH IMAGE PROCESSING

APPLICATIONS

PHAM MINH HUNG

ADVISOR: PhD DANG TRI DUNG STUDENT: TRAN CHI THANH

GRADUATION THESIS MECHATRONICS ENGINEERING TECHNOLOGY

HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION

FACULTY FOR HIGH QUALITY TRAINING

GRADUATION THESIS

PHAM MINH HUNG 18146025

 HO CHI MINH CITY – 02/2023 

MAJOR OF MECHATRONICS ENGINEERING TECHNOLOGY

RESEARCH AND IMPLEMENT A MODEL OF MANGO SORTING MACHINE WITH IMAGE PROCESSING

1 Title of Capstone Project:

RESEARCH AND IMPLEMENT A MODEL OF MANGO SORTING MACHINE WITH IMAGE PROCESSING APPLICATIONS

2 Initial materials:

• Calculation of data, design and construction of hardware

• Calculation and selection of suitable engines

• Control software design

• Apply the image processing for classification

Dean Head of Department Advisor

HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY

AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING

Capstone Project Proposal

Student name: Tran Chi Thanh Student ID: 18146060 : 0938 669 029 Student name: Pham Minh Hung Student ID: 18146025 : 033 376 4601 Advisor: PhD Dang Tri Dung : 0967 360 960

ADVISOR’S EVALUATION SHEET

Student name: Student ID:

Student name: Student ID:

Major:

Project title:

Advisor:

EVALUATION 1 Content of the project:

2 Strengths:

3 Weaknesses:

4 Approval for oral defense? (Approved or denied)

5 Overall evaluation: (Excellent, Good, Fair, Poor)

6 Mark:………/100(in words: )

Ho Chi Minh City, January, 2023 ADVISOR (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -

Ho Chi Minh City, January, 2023

PRE-DEFENSE EVALUATION SHEET

Student name: Student ID:

Student name: Student ID:

Major:

Project title:

Reviewer:

EVALUATION 1 Content and workload of the project

2 Strengths:

3 Weaknesses:

4 Questions:

5 Approval for oral defense? (Approved or denied)

6 Overall evaluation: (Excellent, Good, Fair, Poor)

7 Mark:…………/100(in words: )

Ho Chi Minh City, January, 2023 REVIEWER (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -

Ho Chi Minh City, January, 2023

EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER

Student name: Student ID:

Student name: Student ID:

Major:

Project title:

Name of Defense Committee Member:

EVALUATION 1 Content and workload of the project

2 Strengths:

3 Weaknesses:

4 Overall evaluation: (Excellent, Good, Fair, Poor)

5 Mark:…………/100(in words: )

Ho Chi Minh City, January, 2023 COMMITTEE MEMBER (Sign with full name) THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -

Ho Chi Minh City, January, 2023

PREFACE

Students will use their knowledge of design, mechanics, computation, electrical and electronic circuits, and programming techniques to research and create models for their graduation project As a result, this is the point at which students have a more realistic view of the knowledge they have gained and can put it to use in the creation of products

The group's assigned topic is to create a model for categorizing mangoes using camera operates on a miniature conveyor Our understanding of machine detail design is still limited due to our limited experience, so we have done our best to refer to relevant documents and lectures, but errors are unavoidable in design and construction calculations, design, etc Looking forward to the guidance and dedicated guidance of teachers to help students like us best complete this graduation project

Our students, Pham Minh Hung and Tran Chi Thanh, are certain that this is

a project that we will complete on our own with the assistance of our instructors and other dedicated teachers

Team members rely solely on prior literature and online research, and we hereby declare that all figures are accurate and have not been plagiarized from any previous material or work

Ho Chi Minh City, 02/2023 Student implementation

Tran Chi Thanh Pham Minh Hung

ACKNOWLEDGEMENTS

The project team would like to especially thank PhD Dang Tri Dung was the one who guided and helped the group during the implementation of the project, gave research directions, answered questions and enthusiastically stood by the group

During the implementation of the group, the teachers have absorbed practical knowledge and serious and effective working methods The group would like to thank the teachers of Mechatronics Department for their support students with solid basic knowledge and create the best conditions for students in the learning and research process

The most important, unspoken support of family and friends is always the motivation for the whole team to work hard and complete the project in the best way possible

Once again, the team is very honored to be a student of Ho Chi Minh City University of Technology and Education and a student of dedicated teachers, this thank you is also the team's deep gratitude to the team teachers, relatives and friends

COMMENT OF THE ADVISOR

………

………

………

………

………

………

………

………

………

………

………

………

………

………

………

………

………

Ho Chi Minh City, 06/2022

Advisor

PROJECT IMPLEMENTATION SCHEDULE

Advisor

10/09/2022

Meet the lectures to listening about the request

of topic to make project and approves topic which student has selected Student write out the project outline and schedule

TABLE OF CONTENTS

CHAPTER 1: INTRODUCTION 1

1.1 Overview of the subject 1

1.1.1 An overview of global mango production 1

1.1.2 The global mango import market 2

1.1.3 Vietnam's mango production and export situation 4

1.1.4 Vietnam's mango growing and harvesting process 6

1.2 Motives for selecting the topic 7

1.3 Attained objectives and outcomes 8

1.4 Scope and object of research 8

CHAPTER 2: THEORETICAL MATERIAL 10

2.1 Image processing application for mango classification 10

2.1.1 What exactly is image processing? 10

2.1.2 What exactly is an Embedded System? 13

2.1.3 Image processing is used to identify mangoes based on color 15

2.2 Theoretical underpinnings of hardware design 16

2.2.1 Conveyor belt operation principle 16

2.2.2 The camera's operational principle 18

2.2.3 The principle of operation of a infrared proximity sensor 20

2.2.4 The operation of a geared motor 21

2.2.5 The operation of a servo motor 23

CHAPTER 3: DESIGN AND CONSTRUCTION 26

3.1 Design of Hardware 26

3.1.1 Design models on drawings 26

3.1.2 Block diagram of hardware 29

3.2 Electrical circuit design 33

3.3 Equipment are used in project 37

3.3.1 Box steel is made to conveyor frame 37

3.3.2 Belt conveyor 37

3.3.3 Webcam 720P Genius 1000X 38

3.3.4 Controller circuit Arduino Nano 39

3.3.5 Infrared proximity sensor E18-D80NK 41

3.3.6 Geared Motor with Encoder GB37-545 42

3.3.7 Servo Motor MG995 43

3.3.8 Module relay 5V 1 channel 44

3.3.9 Module DC Circuit LM2596 45

3.3.10 Power Adapter 12V 5A 46

3.4 Model construction 50

CHAPTER 4: CLASSIFICATION OF MANGOES BY COLOR AND SIZE USING IMAGE PROCESSING TECHNOLOGY 51

4.1 Overview of image processing technology: 51

4.1.1 Introduction image processing 51

4.1.2 Basic problems in image processing 51

4.2 Introduction to the Python programming language and Tensorflow library 54

4.2.1 Python Programming 54

4.2.2 Tensorflow library 54

4.3 MoblilenetV2 model: 55

4.3.1 Overview of MoblilenetV2 model 55

4.3.2 MoblilenetV2 Application: 56

4.3.3 Mobilenet V2 Function: 56

4.4 Edge detection method and color recognition 57

4.4.1 Method of detecting the edge of an object 57

4.4.2 Color recognition method 60

4.5 Mango classification system using machine vision 65

4.5.1 Block diagram 65

4.5.2 Block listing of processing: 66

4.5.3 Mango classification system using image processing technology 67

4.5.4 Classification Process: 68

4.5.5 Size of the mango and Experiment: 82

CHAPTER 5: CONCLUSION AND FUTURE WORK 92

5.1 What we have done in this project: 92

5.2 Difficulties: 92

5.3 Future work: 92

REFERENCES 93

TABLE OF FIGURES

Figure 1: Mango production worldwide from 2000 to 2021 2

Figure 2: From 2014 to 2021, the top ten importers of Fresh Mango (Unit: USD) 4

Figure 3: Vietnam's export mango [3] 5

Figure 4: Introduction to image processing [4] 10

Figure 5: Image processing procedures [5] 12

Figure 6: Introduction embeded system [6] 13

Figure 7: Block diagram of an embeded system [7] 14

Figure 8: Applications of embeded system [8] 15

Figure 9: Steps for performing image-based mango classification 16

Figure 10: Diagram of the belt conveyor's operation [9] 17

Figure 11: Pixels [10] 18

Figure 12: Digital camera sensor [11] 19

Figure 13: Working principle of infrared sensor [12] 21

Figure 14: Structure of the geared motor [13] 23

Figure 15: Brushed and brushless servo motors [14] 24

Figure 16: Principle working of DC servo motor [15] 25

Figure 17: 2D drawings and projections of the model 26

Figure 18: Assembly drawing of the machine……….27

Figure 19: Conveyor drawing………27

Figure 20: 3D model in front view………28

Figure 21: 3D model in back view………28

Figure 22: Design of electrical circuits………33

Figure 23: Algorithm diagram of electrical system……… 34

Figure 24: Specification of rectangular box steel [16]……….37

Figure 25: Belt conveyor 1mm [17]……….37

Figure 26: Module webcam 720P Genius 1000X [18]……….38

Figure 27: Arduino Nano Board [19]……… 39

Figure 28: Aruino Nano Pinout Source: eTechnophiles.com [20]………40

Figure 29: Proximity sensor E18-D80NK [21]……… 41

Figure 30: Geared Motor with Encoder GB37-545 [22]……….42

Figure 31: Image of servo motor MC955 [23]………43

Figure 32: Module relay 5V 1 channel [24]………44

Figure 33: Module DC Circuit LM2596 [25]……….…45

Figure 34: Adapter 12V 5A supply power [26]……….……….46

Figure 35: Model of mango sorting machine by color based on image processing after completion………50

Figure 36: Machine vision……… 51

Figure 37: MobilenetV2 Network Layer……… 56

Figure 38: Non-maximum Suppression………59

Figure 39: Threshold filtering………60

Figure 40: RGB color space……… 60

Figure 41: CMYK color space………61

Figure 42: HSV color space……… 62

Figure 43: CIE LAB color space……… 62

Figure 44: H-S Color Strip……….63

Figure 45: HSV table……… 64

Figure 46: Block Diagram of the sort mango machine……….65

Figure 47: Diagram of the mango classification process using image processing technology……….67

Figure 48: Capture Real Mango Code……….…69

Figure 49: Ripe Mango……….… 69

Figure 50: Semi Ripe Mango……….70

Figure 51: Unripe Mango……… 70

Figure 52: Gaussian function………71

Figure 53: Gray image with Gaussian blur result……….71

Figure 54: Threshold function……… 72

Figure 55: Threshold image……… 72

Figure 56: Import Tensorflow library……… 73

Figure 57: Network structure of MobilenetV2………75

Figure 58: MobilenetV2 model design……….75

Figure 59: Distributor equation………76

Figure 60: Training console……… 76

Figure 61: Training Parameter……….77

Figure 62: Training Loss and Accuracy……… 77

Figure 63: Print MobilenetV2 Model………77

Figure 64: Training model of MobilenetV2 results………78

Figure 65: Predict Function……… 78

Figure 66: Prediction function……… 79

Figure 67: Training Accuracy Parameter………79

Figure 68: Prediction of 10 different ripe mango graph……… 81

Figure 69: Prediction of 10 different semi ripe mango graph……….81

Figure 70: Prediction of 10 different Unripe mango graph……….81

Figure 71: Length of the mango (11cm)……….82

Figure 72: Width of the mango (7cm)……… 83

Figure 73: Capture of code application length and width………84

Figure 74: Pixel value of mango size………84

Figure 75: Bounding area of a mango……….84

Figure 76: Lenght error graph for ripe mango……… 87

Figure 77: Lenght error graph forripe mango………87

Figure 78: Width error graph for Semi ripe mango……… 88

Figure 79: Length error graph for Semi ripe mango………88

Figure 80: Width error graph for unripe mango………89

Figure 81: Length error graph for unripe mango……… 89

Figure 82: Unripe Mango Results………90

Figure 83: Ripe Mango Results……….90

Figure 84: Semi Ripe Mango Results………91

LIST OF TABLE

Table 1: Explain the significance of the blocks in hardware diagrams……….…30

Table 2: Explanation of electrical part block diagram……….…36

Table 3: List of device funciton……….49

Table 4: Block listing of processing……….….67

Table 5: Camera Settings……… 69

Table 6: Network structure of MobilenetV2……….… 73

Table 7: Prediction value of Ripe mango……….… 79

Table 9: Prediction value of Semi ripe mango……….… 80

Table 9: Prediction value of Semi ripe mango……….… 80

Table 10: Ripe Mango size data collection……….…85

Table 11: Semi ripe Mango size data collection……….85

Table 12: Unripe Mango size data collection……….85

Table 13: Experiment in 10 ripe mango……… 86

Table 14: Experiment in 10 semi ripe mango……….…86

Table 15: Experiment in 10 unripe mango……… … 86

1

CHAPTER 1: INTRODUCTION 1.1 Overview of the subject

1.1.1 An overview of global mango production

Mango is a tropical fruit tree that is grown in over 90 countries worldwide, covering an area of approximately 1.8-2.2 million hectares Mangoes are popular, with the second highest consumption of any fruit, trailing only bananas From

2001 to 2010, global mango production more than doubled Mango production in

2014 was approximately 28.8 million tons, accounting for 35% of global tropical fruit production, of which approximately 69% was mango Asia Pacific (India, China, Pakistan, the Philippines, and Thailand) accounts for 14% of total production, with Latin America and the Caribbean (Brazil and Mexico) accounting for 14% and Africa accounting for 9% Mango production in developed countries (the United States, Israel, and South Africa) is approximately 158,000 tons

Mango production in Asian countries ranges from 13 to 17 million tons per year, with India producing 13 to 17 million tons, China producing 4 million tons, Thailand producing 2 to 5 million tons, and Pakistan producing 1.7 million tons Mexico leads the Americas with an annual output of about 1.5 tons, followed by Brazil with 1.2 tons Nigeria and Egypt are Africa's two largest mango producers

The popularity of processed mango is growing day by day Mango juice, pickled mangoes, mango chutney, mango pulp, mango paste, and mango concentrate are examples of processed mango products puree), dried mango fruit, brined mango slices, and mango flour India is the largest processed mango exporter, followed by Pakistan, Brazil, and Zimbabwe The United Arab Emirates, Saudi Arabia, Kuwait, the United States, and Canada are the countries that import this type

2

Figure 1: Mango production worldwide from 2000 to 2021

(in million metric tons) Source: FAO [1]

1.1.2 The global mango import market

Mexico, Brazil, Peru, Ecuador, and Haiti are the primary suppliers to the North American market in terms of distribution The West Asian market is dominated by India and Pakistan Mangoes are sold to Southeast Asian countries

by the Philippines and Thailand Mangoes are purchased by the European Union from South American and Asian countries The United States is the world's largest mango importer Mexico, Peru, Ecuador, and Brazil are the primary suppliers of fresh mangoes to the United States, with Mexico accounting for 60.8% Brazil, Peru, and Ecuador have increasingly competed with Mexico in exporting mangoes

to the United States at the start and end of the season over the last five years The United States also re-exported a small amount to other countries, primarily Canada and the United Kingdom

The most popular mango varieties for export continue to be Kent, Tommy Atkins, Haden, and Keitt, which are red, have less fiber, firm flesh, and are better

3

suited for long-distance transportation than other varieties Blue-skinned varieties like Ataulfo and Amelie have only recently gained international traction Alphona, Dudhpeda, Kesar, Sindhu, Pairi, Desi, Chaunsa, Langra, and Katchamita are some

of the other varieties The majority of the later emerging varieties originated in India and Pakistan

The popularity of processed mango is growing Mango juice, pickled mangoes, mango chutney, mango pulp, mango paste, and mango concentrate are examples of processed mango products puree), dried mango fruit, brined mango slices, and mango flour India is the largest processed mango exporter, followed

by Pakistan, Brazil, and Zimbabwe The United Arab Emirates, Saudi Arabia, Kuwait, the United States, and Canada are among the countries that import this type

China, the United States, and the European Union have the world's highest demand for mangoes Mangoes are primarily imported into the United States from Brazil, Peru, and Ecuador Latin America, Central America, Africa, Israel, Pakistan, India, and Spain are among the countries that supply mangoes to Europe Mango season in Europe: autumn/winter for Brazil and Peru, spring for West Africa (Burkina Faso, Mali), summer/autumn for the Middle East, Central America, Asia, and West Africa (Israel, Egypt, Pakistan, Costa Rica, Mexico, Dominican Republic, Senegal) The Netherlands is Europe's largest mango importer The United Kingdom and France are also important markets

China's mango growing area is expected to reach 349 thousand hectares by

2020, with a total output of 3,306 million tons However, the country also imports

a lot of mangoes and mango products Mangoes are primarily imported by China from Thailand, Taiwan, Vietnam, Australia, and Peru More than 90% of China's total mango imports are from these five countries However, in recent years, there has been a significant increase in the importation of mangoes from Vietnam In

4

addition to fresh mango imports, China imports far more mango juice than it exports

1.1.3 Vietnam's mango production and export situation

Mangoes are grown in Vietnam from the south to the north, with the majority grown in the Mekong Delta provinces of Tien Giang, An Giang, Dong Thap, Vinh Long, and Can Tho, Ben Tre Mango is also grown in the provinces

of Khanh Hoa, Dong Nai, Ba Ria Vung Tau, Son La, Lang Son, and the Red River Delta The country's mango growing area is approximately 87,000 hectares, with

an annual output of more than 969,000 tons Vietnam ranks 13th in the world in terms of mango production, but exports remain modest and fall outside the top ten mango exporting countries

Vietnam is the world's 13th largest mango producer; mango is one of the most important tropical fruits grown in Vietnam, second only to banana with a total planting area of more than 87,000 hectares in the country; in 2020, total mango production in Vietnam reached 893.2 thousand tons, a 6.5% increase over the same period last year Mango is grown the most in the Mekong Delta region,

Figure 2: From 2014 to 2021, the top ten importers of Fresh Mango (Unit: USD)

Source: Tridge [2]

Figure 3: Vietnam's export mango [3]

b Basis for aggregation

Harvested mangoes are usually taken to a collection facility near the mango orchard The harvested mango is removed from the bag for the first time All damaged or defective fruit is discarded, and fruit that does not meet export market criteria is separated for domestic sale

Mangoes are transported to the packing facility in covered containers

Mangoes are shipped directly from the garden to the packer or processing facility where there is no agglomeration facility At that point, the new mango is removed from the bag and the initial screening is performed

Mangoes are then classified based on their size, weight, variety, maturity, and uniformity

d Packaging, warehousing, and distribution

For export, mangoes are typically packed in plastic or cardboard boxes Mangoes will be labeled after packaging for quality assurance

7

Mangoes are kept cold (around 13 degrees Celsius) until they are loaded into trucks or refrigerated containers and transported to ports or airports for export

1.2 Motives for selecting the topic

According to the data presented above, the global mango consumption market

is enormous Furthermore, our Vietnam is a mango-producing country with a large export potential for this fruit The issue here is how to shorten the process so that the mango is as fresh as possible in order to increase market competitiveness As

a result, rapid and accurate sorting of mangoes in large quantities is an extremely important step in increasing both export and value brought

However, in Vietnam, mango sorting is currently done primarily by hand by unskilled laborers Mangoes are graded during this process based on factors such

as ripeness, size, and shape, among others Through the subjective evaluation of people based on their senses This is a time-consuming and labor-intensive task Furthermore, the classification of mangoes based on human vision frequently has limitations such as low objectivity, low working productivity, and inability to meet classification requirements that require a high degree of accuracy To be able

to export to foreign countries, particularly fastidious markets, accuracy and high homogeneity are required

Machine vision and image processing technology has been increasingly developed and applied in most fields of science and social life in recent years, particularly in replacing the role of humans in automated sorting and monitoring chains With the advent and widespread popularity of industrial conveyors, we can optimize almost any field, including agricultural product processing and sorting Industrial conveyors can reduce errors, labor, and bring continuous working time and efficiency by automating at all stages and utilizing technology high work productivity

8

The research team would like to introduce the topic: "RESEARCH AND IMPLEMENT A MODEL OF MANGO SORTING MACHINE WITH IMAGE PROCESSING APPLICATIONS" with the goal of being able to create a suitable mango classification system that is user-friendly, high operability, and affordable and do With this model, we can easily develop into a complete system that can assist farmers Using, operating, adjusting, and replacing manual labor assists in reducing labor force, lowering costs, and increasing work productivity

1.3 Attained objectives and outcomes

In fact, the mango sorting line is used with a variety of criteria such as size, shape, color, and so on With the topic's limitations and a focus on the algorithm, the team members designed and built a model of sorting mangoes by color using

an Arduino microprocessor to control with a simple structure as follows:

• The frame is typically made of galvanized steel, stainless or profiled steel, or aluminum

• Conveyor surface depending on the application, a variety of conveyors and materials can be used

• A camera module-based optical proximity sensor is available on the market

• Using a DC electric motor to rotate the conveyor and a stepper motor to drive the sorters

• Make use of the well-known Arduino microcontroller

1.4 Scope and object of research

The object of implementation for the topic "RESEARCH AND IMPLEMENT

A MODEL OF MANGO SORTING MACHINE WITH IMAGE PROCESSING APPLICATIONS" will be to create a model with an image processing application

to classify fresh mangoes with standards The scope of research and implementation includes the following:

9

• Content 1: Discover existing research on the topic, specific product is fresh mango that is already on the market, research related equipment and documents

• Content 2: Understand the theoretical foundations of the topic, including: standards for classifying fresh mangoes, conveyor operating principles, camera operating principles, how to calculate and select appropriate parameters, raw materials operation of sensors and actuators

• Content 3: Motor control protocol, mechanical and circuit design, hardware-software linkage, image processing, software embedding in hardware

• Content 4: Learn about Arduino programming and how to write programs,

as well as training model algorithms

• Content 5: Using survey results, design, calculate, and build, then test and evaluate the results

10

CHAPTER 2: THEORETICAL MATERIAL

2.1 Image processing application for mango classification

2.1.1 What exactly is image processing?

What exactly is image data?

The amplitude of F at any pair of coordinates (x, y) is called an image An image is defined as a two-dimensional function, F(x, y), where x and y are the spatial coordinates is the image's intensity at that point A digital image is one that has finite x, y, and amplitude values for F In other words, an image can be identified by a two-dimensional array with rows and columns

The digitized image is made up of a finite number of elements, each with a unique value at a specific location These are known as picture elements (picture elements, image elements or pixels, where pixel is the most common term)

Images of various types

• Binary Image: As the name implies, a binary image contains only pixels with values of 0 or 1, where 0 represents black and 1 represents white Monochrome is another name for this image

Figure 4: Introduction to image processing [4]

11

• 8-bit color image: The most well-known image format, with 256 different shades of color, is also known as a grayscale image In this format, 0 represents black, 255 represents white, and 127 represents gray

• 16-bit color image: This is a high color image format with 65,536 different colors The color distribution in this format differs from that

of a Grayscale image, which is typically divided into three channels

of Red, Green, and Blue

What exactly is image processing?

Image processing is the process of converting an image into a digital format and then performing various operations on that image to extract useful information When using a predefined signal processing method, image processing systems typically treat all images as 2D signals

The following are the primary types of image processing:

• Recognize - Identify or detect objects in images

• Sharpen and Restore - Enhance images created from originals

• Pattern Recognition - Determine different patterns surrounding objects in

an image

• Retrieve - Browse and search images similar to the original image in a large database of digital images

Steps for basic image processing

• Image acquisition: The first step in image processing, also known as preprocessing, is image acquisition It entails obtaining an image from a source, which is usually hardware-based

• Image enhancement is the process of bringing out features in an image that have been obscured by changing the brightness, contrast, and so on

• After an image has been segmented into regions, each region is represented and described in a format suitable for further processing The presentation addresses image characteristics as well as area properties The extraction of quantitative information that distinguishes one feature class from another is referred to as description

• Identity: Identity gives an object a label based on its description

Figure 5: Image processing procedures [5]

13

2.1.2 What exactly is an Embedded System?

These are systems that combine hardware and software to solve specialized problems in a variety of industrial fields, including control automation, monitoring, and communication The features of embedded systems include stable operation and high automation

Embedded systems are typically designed to perform a single task An embedded system is a computer that performs only one or a few specific functions and includes specialized hardware and machinery not found in a general-purpose computer in general terms

Embedded systems have a high level of automation and are often designed with separate functions A system that can only perform one or a few fixed functions required by the devices cannot be universally useful They can be optimized in terms of size and cost of production

Embedded systems are frequently produced in a wide range of series and in large quantities Embedded systems are simple for a microcontroller but more

Figure 6: Introduction embeded system [6]

14

complex with many units and peripherals This system is commonly used in devices such as digital clocks, MP3 players, and traffic lights

Features of an embedded system

Embedded systems, as opposed to multi-function computers, are specifically designed to perform one or more tasks The system must operate in real time, be secure, and be applicable

Some systems do not necessitate stringent constraints Allows systems to reduce production costs by simplifying hardware Some systems also necessitate complexity in the devices they control

An embedded system is typically not a standalone unit, but rather a complex system embedded within the device it controls Embedded systems are written in firmware software and stored on ROM and flash memory chips, and they operate with limited hardware resources

Embedded systems are now present in many products that have a direct impact on human life:

• Satellites, air traffic control systems, and global positioning systems

• Refrigerators, microwaves, and ovens are examples of household appliances

Figure 7: Block diagram of an embeded system [7]

15

• Routers, hubs, gateways, and so on are examples of networking devices

• Copiers, fax machines, printers, scanners, and other office equipment

• Osmosis machines, pacemakers, and other medical devices

• Answering machines that work automatically

• Robots and automated production lines in industry

2.1.3 Image processing is used to identify mangoes based on color

Successfully researching and developing an embedded system model to classify mango fruit quality based on the results, as well as overcoming the limitations of previous studies, has significant implications in the development of mango fruit The system can handle Vietnam's mango export classification requirements

When compared to manually sorting mangoes, it requires a lot of effort and time, but the efficiency is low Since then, the combination of image processing and embedded systems has served as the foundation for developing a tool to help classify mangoes more accurately and efficiently

Figure 8: Applications of embeded system [8]

16

2.2 Theoretical underpinnings of hardware design

2.2.1 Conveyor belt operation principle

a What is the conveyor belt?

Conveyor is simply understood as a mechanical machine used to transport objects from one point to another, from position A to position B Instead of transporting products with workers that waste time, labor costs create a messy working environment, the conveyor belts can solve that

It saves labor force, number of workers, reduces time and increases labor

Figure 9: Steps for performing image-based

mango classification

17

productivity Therefore, the conveyor belt is one of the important parts in the production and assembly line of these factory Contribute to creating a modern and scientific production environment and free up labor force bring high economic efficiency for companies, factories

b Structure of an industrial conveyor

• A screw motor and speed controller control

• Roller set, active transmission

• Roller support system

• Belt or roller system

c The principle of the conveyor belt

When the roller actively rotates, causing friction between the roller and the belt conveyor to move When the belt is coincident with the conveyor roller, we adjust the passive roller or belt tension system so that the belt tension creates friction between the belt and roller make the first move This friction force will cause the conveyor to move indefinitely Rollers placed below the conveyor surface (belt support rollers) are used to prevent belt sagging, which also reduces friction on the conveyor path

Figure 10: Diagram of the belt conveyor's operation

[9]

18

2.2.2 The camera's operational principle

Every camera is essentially a sealed, light-proof box with one end open to allow light in and the other end housing a film or image sensor It generates an image by collecting light rays reflected from the scene and projecting them onto

a light sensitive film surface or an image sensor These fundamental components are as follows:

• Light is captured by a single lens

• A viewfinder system: this will allow us to see the frame that will be retracted

• A focus element: controls the positioning of the lens layers that produce sharp images

• One shutter: to time when light enters the film/image sensor

• The aperture in the lens controls the amount of light that passes through it

• A dark box in which a film or image sensor is mounted and receives only the impact of light passing through the lens

Figure 11: Pixels [10]

With a multicolor filter, the pixels are arranged and arranged in a mosaic form (mosaic style) This mosaic is made up of blocks (each block has four pixels), two green, one red, and one blue Because each pixel is only sensitive to its own color, the image is displayed in red, green, and blue shades scattered throughout the mosaic The camera's processor or editing software must be capable of producing accurate color tones and smooth image rendering

The image is converted into digital data after being recorded from a digital camera sensor and passed through an amplifier This is what an analog-to-digital converter does

Most modern cameras convert to 16-bit but only use 14-bit, allowing for more flexibility in post-processing and filtering In which case, 14 bits correspond to

Figure 12: Digital camera sensor [11]

2.2.3 The principle of operation of a infrared proximity sensor

a What is an infrared sensor ?

Passive Infrared, also known as IR Sensor, is the English name for an infrared sensor An electronic device that measures and detects infrared radiation

in its surroundings Because the wavelength of an infrared sensor is longer than that of visible light As a result, the infrared sensor emits rays that are visible to the naked eye

Infrared radiation is emitted by anything that generates heat Infrared radiation, also known as heat radiation, has a longer wavelength than light It will also be shorter in duration than microwave radiation Beyond the red wave is infrared Red is the color with the longest wavelength in natural light Objects with

a temperature greater than 35°C will normally emit infrared wavelengths An astronomer named William Herschel discovered infrared radiation by accident in

1800 He discovered that the temperature beyond red light was the highest while measuring the temperature of each color of light (separated by a prism)

b Sensor classification for infrared

Infrared sensors are classified into two types: active and passive sensors Infrared sensors generate and detect infrared radiation A light emitting diode (LED) and a receiver are typically used in active infrared sensors When an object approaches the sensor, infrared light from the LED is reflected off it The receiver will then detect them

21

A passive infrared sensor only detects infrared rays emitted by other objects such as people, animals, or heat sources However, it does not emit any infrared rays on its own After detecting the heat source, the sensor will analyze to determine the alarm condition That is why it is referred to as passive They only detect infrared rays, not their source

c The infrared sensor's operation principle

Infrared sensors detect specific wavelengths of light in the infrared (IR) spectrum using a specific light sensor By using LEDs to generate light of the same wavelength as the sensor The intensity of the received light is then visible When an object is in close proximity to the sensor, the light from the LED bounces off it and enters the light sensor This causes a rather large magnitude jump

2.2.4 The operation of a geared motor

a What is a geared motor ?

The geared motor is made up of two components: the electric motor and the gear box The stator and rotor are the two main components of an electric motor The stator is made up of three-phase electricity windings wound on iron cores Set

on a circular rim A rotating magnetic field is formed as a result The cylindrical rotor functions as a coil wound on a steel core

The gear shifter, screw, and other components are housed within the reducer

to slow the rotation speed The angular instantaneous velocity vector is reduced

Figure 13: Working principle of infrared sensor [12]

22

using this box Increases torque and acts as a link between the electric motor and the rest of the system The reducer's other end is connected to the load

b What is the basic function of a gear reducer ?

The geared motor's function is to slow down and slow down the rotation The geared motor is a constant gear ratio direct-coupled drive mechanism Direct-coupled actuators control the braking and deceleration of revolutions and devices Has a fixed motion ratio and is used to constrain the angular instantaneous velocity vector Increase torque at the same time This is also the part that connects the motor reducer to the part that drives the machine to work

c What is a speed reducer gear ?

A mechanical device, the reducer Inside the reducer, there are numerous gears, screws, and other components that work together to reduce the rotational speed of the electric motor (however, the speed here is not the long speed)

The reducer works like a direct-coupled transmission mechanism, with a constant gear ratio to reduce angular velocity and increase torque The reducer is installed between the electric motor and the load, with the core of the electric motor connected to the reducer and the core of the reducer connected to the load (chain, belt, hard connection)

d The basic operation of the motor gear reducer speed

A similar principle governs the operation of geared motors Specifically, when we want the number of revolutions of the reduction gearbox's output shaft

to be low Then, when we add a reduction gearbox to the electric motor, we only spend a little money From there, the number of shaft revolutions can be changed more freely

Electric motors are electrical devices that run on electricity There are two types of electric motors: 1-phase AC motors and 3-phase AC motors Electrical

23

energy is converted into mechanical energy by the electric motor It then assists

in the operation of equipment and machines such as conveyor belts, water pumps, electric fans, cranes, and so on Most current electric motor lines comply with IE2, IE3,

2.2.5 The operation of a servo motor

a System of servos

Servo is a closed-loop feedback control drive system that receives signals and executes commands from PLCs quickly and accurately A servo set consists of a servo controller (servo drive), a servo motor, and an encoder to provide feedback from the motor to the controller Servos are used for precise position control, torque adjustment to meet the needs of various applications, and extremely fast speed changes (response in ms)

The servo motor is classified as a component of the servo system When operating at extremely high speed and accuracy, the servo motor receives the signal from the controller and provides the necessary movement force for the mechanical devices

There are two types of servo motors: AC servo motors and DC servo motors

AC servos are used in industrial machinery because they can handle higher

Figure 14: Structure of the geared motor [13]

or for general industrial and civil purposes

Brushless direct current motor: They are typically small in size and weight However, because of its smooth and stable torque, this type of motor responds quickly and has a low inertia Brushless DC servo motors are more difficult to control, but they work smarter due to flexible electronic switching in square wave

or sine wave switching The motor has the advantages of being energy efficient, emitting little electromagnetic radiation, and requiring little maintenance

Figure 15: Brushed and brushless servo motors [14]