Basic programming for electronics final project report project automatic real time cutter based on vision

We need to choose camera expected length, allowed error, camera distance , we set up and focal length of the camera some function we will explain later.. Then in the pink palette, users

PHENIKAA UNIVERSITY

FALCULTY OF ELECTRICAL ELECTRONIC ENGINEERING

-BASIC PROGRAMMING FOR ELECTRONICS

FINAL PROJECT REPORT

Lecturer: Dr.Minhhuy Le

1 Vu Phuong Thuy

2 Pham Thi Thu Hien

3 Tran Thi Thanh Thuy

WORKLOAD DISTRIBUTION

o Write report

o Make Powerpoint

o Edit video

o Present software of Cutter tab

o Present hardware of Cutter tab

o Present Manage tab

TABLE OF CONTENTS

SECTION 1: PREFACE 2

SECTION 2: INTRODUCTION 3

SECTION 3: PROJECTS DETAILS 4

1 G ENERAL INTRODUCTION 4

1.1 Some video links 4

1.2 Idea 4

1.3 Software interface 4

1.4 Hardware 7

2 P ROJECT DETAILS 8

2.1 Cutter 8

2.1.1 Coding 8

Image Conversion from Color to Binary Image (Fig 6.1) 9

Binary Image Conversion (Fig 6.2) 10

Fill Hole (Morphology) (Fig 6.3) 10

Removing Small Objects (Fig 6.4 ) 11

Particle Analysis (Figur 6.5 ) e 11

Calculate real length of the object (Fig 5.2) 12

Arduino Code (Fig.8) 14

2.1.2 Design and Operation 14

Operation 16

2.2 Manage application 18

STEP 1 Get information and list into queue(13.1) 19

STEP 2: Check receive and calculate total revenue(5.2) 22

STEP 3: Write data to file 25

SECTION 4: DISCUSSION 25

SECTION 5: REFERENCES 26

2 | P a g e

Section 1: PREFACE

“First of all, we would like to thank Phenikaa University for introducing Basic Programming for Electronics into the curriculum This subject gives us much knowledge about control programming, software as well as hardware control The course also helps us improve many soft skills: presentation skills, report writing skills, teamwork skills, etc These are extremely useful knowledge for students and our future work

In particular, we would like to express my deep gratitude to the subject lecturer - Dr

Le Minh Huy for teaching and imparting valuable knowledge to us during the past study period, he helped us to expand our knowledge of control programming in the field that we are studying, prepared lessons and plans so that students can absorb the lesson most effectively even though this subject is highly rated as very hard We also want to say thank you to Mr Nguyen The Thao for his help from the beginning, Thuy’s father for making our machine from our sketch and Mr Tran Van Quang for his help too

We have tried our best to complete this report However, because our knowledge is still limited and our ability to absorb reality is still confusing, the project certainly cannot avoid shortcomings and many gaps Hoping you can look at the strengths and ignore some of the weaknesses of this project Of course, we hope you will consider and comment to improve our project

Thank you sincerely!"

Section 2: INTRODUCTION

Automation is increasingly being used for industry and business, particularly since technological advances have increased their range However, it is just applied in big industries or businesses Small business households find it difficult to access technical development because they can not afford operational costs It becomes a big barrier to expanding the automation range

The aim of this project was to develop an automatic real-time cutter based on their length and a small application to manage client information The based knowledge described here is LabVIEW Vision, basic knowledge, and Arduino control At the core

of the system is the vision part It measures the length of objects measured by the vision system in real-time then controls hardware by Arduino code Besides, there is a manage client information app to make management easier

The report begins with a detailed description of the project: idea, hardware and software, design and operation The discussion about results of laboratory and execution are then presented and analyzed Finally, the limitations of the project are discussed and there is some recommendation for future work

4 | P a g e

Section 3: PROJECTS DETAILS

1 General introduction

1.1 Some video links

Before st ting, to understand our project easily, you can w ch some our videos: ar at

1.2 Idea

In this project, we do experiments cutting rice pipes (a kind of snack was made of rice)

A member of our team finds it difficult to cut rice pipe with scissors and can not do it for a long time Therefore, we think carefully about something that can make it automatic, easier Gradually, we build this idea then make it real After finishing, we continue on developing a small application to manage

1.3 Software interface

There are two tabs: Cutter and Manage

Fig.1 Cutter tab

Cutter tab (Fig.1) is used to control cut task Users will interact with green palette

We need to choose camera expected length, allowed error, camera distance , we set up and focal length of the camera (some function we will explain later) Then in the pink palette, users can get the video recorded from camera, video after processing, number

of particles and especially, we can know the real length of the object and control

hardware to cut when it reaches the expected length

6 | P a g e

Fig.2 Manage tab

Manage tab (Fig.2) is a small application to manage client information Users enter name, kilograms of rice, starting time (the time start to produce) This app will list their name in a queue, then give the amount of money they have to afford and the time they can receive their products In the ‘check receive’ palette, managers only need to enter the client’s name and kilograms of rice when they want to receive their product, then their name will be removed from the queue and list it into the ‘received’ list Finally, you can get the total revenue

We use this part to measure length to control hardware

Fig.5 Main block diagram

Figure 5.1 capture video from camera to do the next task Next, video will be put in to particle analysis processing to measure object in pixel then calculate the real length(5.2) Finally, it will sent requirement to hardware and cut object(5.3) We will introduce more about this code

Let’s see the following flowchart illustrating particle analysis processing

From this, we build particle_analysis_subvi

1

3

2

Fig.6 particle_analysis_subvi

To perform particle analysis, the acquired image needs to be converted to a binary image First, the color image needs to be converted to a grayscale image Second, the grayscale image is converted to a binary image by means of a threshold operation

Image Conversion from Color to Binary Image (Fig 6.1)

IMAQ Extract Single Color Plane function(6.1*) is used to extract luminance plane from color planes In this way, a 32 bit color image is easily converted to 8-bit grayscale image

10 | P a g e

Fig 6.1* Extracting the luminance plane of a color image for conversion to a grayscale image

Binary Image Conversion (Fig 6.2)

A thresholding operation on the image, as seen in Figure 6.2*, is required to change the grayscale image to a binary image Here, the pixels identifying dark objects are converted to image values of 0, whereas the brighter background pixels are converted

to image values of 1 The threshold values for 8 bit gray images range from 0 to 255

In this case, the threshold range slider bar can be used to easily adjust the threshold value By moving the slider bar, we can observe changes to the converted binary image

as the threshold value changes Consider that the threshold value might need to be altered due to a change in lighting conditions

Fig 6.2* Threshold setting

Fill Hole (Morphology) (Fig 6.3)

The Fill Hole function in Figure 6.3* is one of the morphology functions for binary images that can modify objects By using the Fill Hole function, an unfilled object with

a closed boundary can be filled and thus be made more readily available for identification and measurement functions

Fig 6.3* Fill hole

Removing Small Objects (Fig 6.4 )

Fig 6.4* Remove small particles

Figure 6.4* shows the IMAQ Remove Particle function to remove the small objects in a binary image As seen in figure, the iteration numbers are set to 4 Note that the iteration number may need to

be increased if the particle size of unwanted object is larger

Particle Analysis (Figure 6.5 )

Figure 6.5* shows the VIs for particle measurements

Fig 6.5* Particle analysis measurements

In particle measurement output, we can get properties we want In this case, we choose bounding rect left and right

12 | P a g e

Fig.7 Difference between before and after processing

Calculate real length of the object (Fig 5.2)

Back to the main block diagram (Fig.5), after getting left and right coordinate then we multiply it by 0.2 (1 𝑝𝑖𝑥𝑒𝑙 ≈ 0.2𝑚𝑚), we use the following equation to calculate real distance to the left and right coordinate:

𝐹∗ 𝐷

Where 𝑊 is the actual size of the object, 𝑃 is the size of the image of the object reflected on the camera, 𝐹 is the focal length of the camera and is distance from 𝐷camera to object

Finally, we subtract real distance to the right coordinate by the real distance to the left coordinate and we can get the real length of the object

Fig 5.2* Calculate real length of the object (Fig 5.2)

Fig 5.3 Control hardware with a LabVIEW condition by Arduino

Fig 5.3 is code to control hardware with a LabVIEW condition by Arduino We make

an error for the length that user can enter value for this property In LabVIEW, we just write the data into COM3 They are ‘a’ and ‘b’ and connect them with a boolean to control 2 cases If real length is in this range ( 𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝑙𝑒𝑛𝑔𝑡ℎ ± 𝑒𝑟𝑟𝑜𝑟), it will be

‘a’ case, conversely, it will be ‘b’ case

14 | P a g e

Arduino Code (Fig.8)

In Arduino code, we code 2 case, if in case ‘a’, the motor will turn on and it will turn off in case ‘b’

Fig.8 Arduino code

2.1.2 Design and Operation

To do experiment our project, we had the following components

Fig.9 Some our electronic components

The following figure is how we set up everything to do experiment

Fig.10 Experiment setup

We set a camera parallel to the plane The camera can change the distance to the object The machine is put as the above figure

16 | P a g e

The motor is put above the gate and connected with a knife in a point and another point is connected with a spring When the motor turns on, it makes the knife go down If the motor turns off, the spring will pull it up

Operation

After capturing the video, measuring the real length, requirement is sent to arduino to control cutter, we get the result

Fig.11 Moment it cut

As we can see, the expected length entered is 100 mm and error is 10 mm The machine cut when the real length is 100.639 mm It is not bad But the real length is just the app measured, how about the actual length? We measured it Let’s see the following figure

Fig.12 Actual length

As we can see, the actual length is approximately equal to the length that our app

18 | P a g e

2.2 Manage application

To observe generally, you can see the following flowchart and main block diagram

Fig.13 Flowchart and Manage tab block diagram

There are three steps in this part: Get information and list into queue, Check receive

and calculate total revenue, Write data to file

3

1

STEP 1 Get information and list into queue (13.1)

First, we create two arrays then create their local variables to easy to update new element to arrays Then we calculate

kilograms with unit price

Next, we calculate the time that client can get their products To do this, we use

sum_subvi and cal_time_subvi

Fig.14 sum_subvi

Input of this subvi is an array and a numeric and its output is sum array In this case, the array is time array calculated by mutiply kilogram of rice by time per kilogram and the numeric is new time value updated after entering new value and multiplying it with time The subvi will get sum of the last element of array and new value updated

For example, we have an time array [1, 3, 2, 4] so we have sum array is [1, 4, 6, 10],

if we add new element 6 to time array, the subvi will get the last element of sum array and add to 6 is 10 + 6 = 16 Then we get sum array [1, 4, 6, 10, 16]

sum_subvi and the timestamp As the result, we get timestamp array representing the time that client can receive (Fig.16)

Completing the first step, we will get a queue with information of client

Fig.16 Block diagram and front panel of the first step

22 | P a g e

STEP 2: Check receive and calculate total revenue(5.2)

In this step, we will remove information of received client and list it to new list called

‘received’ We use delete_element_array_subvi to remove element Let we explain it

now

Fig.17 delete_element_array_subvi

As you can see, to build this subvi (Fig.17) we use another subvi called

get_index_to_delete_subvi (Fig.18) to get index of deleted element

Fig.18 get_index_to_delete_subvi

In Fig.18, we consider string first, there is a string array and a string representing names list and the name entered We use while loop to get in order element of array and compare it with the name entered whether they are the same or not

Similarly, we compare numeric array representing kilogram of rice with value entered

If two previous conditions are satisfied, the loop will be stopped and returns index (number of loops)

Back to the delete_element_array_subvi (Fig.17), after getting the index, we use Delete From Array function to remove elements of all array we want to delete This subvi returns deleted array To remove information directly in the queue, we use local variable (Fig.19) as we mentioned before Then we create new array to store informations deleted(19.1) (use Insert To Array function) and plot it in waveform graph Finally, we get total revenue by using the previous sum_subvi After calculating sum array of remained cost, we get the last element of this array It is total revenue(19.2)

1

2

24 | P a g e

Let’s observe the difference between before and after removing (Fig.13) When we enter name and kilogram, press ‘enter’ button, information will be sent to received palette

Fig.13 Compare between before and after removing information

STEP 3: Write data to file

This step is very simple We use Write Delimited Spreadsheet function to write data to csv file

Section 4: DISCUSSION

About our cutter tab, it can measure quite good The length that it can measured is approximately the actual length as mentioned before But actually, it also have some limitations There are errors because of some external influences such as: error of measure instrument, the rice pipe is not straight, etc Therefore, the actual length also has error The way we set up laboratory instruments for example camera distance is also a reason why we have error This project start from basic knowledge about LabVIEW Vision, it can not good in all environment As we can see, our background

is a dark plane, don’t have much noise Actually, in some instructions and documents, they use NI Vision Assistant but we can’t use it because of computer display resolution

It is not enough to run Vision Assistant, so we have to make everything manually Therefore, to make it better, we need to do research deeper to apply more advanced vision knowledge In the future, we will try to develop our project apply in other toenvironment and it also works good