Báo cáo nghiên cứu khoa học: Design of two-wheeled self-balancing robot with computer vision

Viet Nam National University, Ha NoiInternational School STUDENT RESEARCH REPORT DESIGN OF TWO-WHEELED SELF-BALANCING ROBOT WITH... The self-balancing two-wheeled vehicle needs to be de

Viet Nam National University, Ha Noi

International School

STUDENT RESEARCH REPORT

DESIGN OF TWO-WHEELED

SELF-BALANCING ROBOT WITH

TEAM LEADER INFORMATION

- Program: Automation and Informatics

- Address: Dong Da, Hanoi

- Phone no /Email: 22070160@vnu.edu.vn

Il Academic results (from the first year to now)

Academic year Overall score Academic rating

Ist year 3.16 Good

2nd year, Ist semester 3.02 Good

HI Other achievements:

- Robotacon WRO 2023 — Consolation prize

- UET Makerthon 2023 — 1st prize

Hanoi, April 17, 2024Advisor Team Leader

(Sign and write full name) (Sign and write full name)

Bui Thanh Ting Nguyễn Thu Hương

1 Project Code:

CN.NC.SV.23 31

2 Member List:

Full name Student ID Email Class

Vũ Huy Khai 22070035 Vuk9461@gmail.com AAI2022A

Pham Tuan Minh 21070741 Minhphamnm@gmail.com AAL2021A

3 Advisor:

Full name: Msc.Bui Thanh Tung

Faculty of Applied Sciences — International school, VNU

1.4 Structure of the Thesis 1 8

2.4 Dynamic model Of DC TmOfOT -.- (c6 E131 13318311 911931 911911 111 1 11v vn 10

2.5 Kinematic model of self-balancing 2-wheeled robot system - - -+-«+ 10

CHAPTER 3 0G 5 5< 9 9 Họ THỌ 0 0000 40060000096 5.0 14

3.1 Declaration, System Analysis, and Controller Design - - «+ ++ss+++s+>+s+2 14

3.1.1 System Parameters nh 15

3.1.2 System Stability AnaÏyS1S - -s- sgk 15

3.1.3 Analysis of System Controllability Based on the principles of automatic controlfor a linear system with a state-space equation as fOILOWS: +5 «<< s+sss+ 16

BALL DETECTION AND TRACKING USING RASPBERRY PI 19

CHAPTER Í, 5 <5 < 5< <9 H HH 040004000400 40896 19

INTRODUCTIOÌN o 5G 5< <5 HH 0000009009609 86 19

CHAPTER 22 5< < 5< 5< << 4 9 9 0050909098900 20

RESEARCH METHODOLOC Y 0 55 5 5 9 9 00 09009009009809804 04 066 20

2.1 Receiving Image From Ca1m€TA - 5 2< 3318833113118 18911 911 118 11 9 1 px rưy 20

2.2 HSV Color Setting nh 5 21

PM 300i 117 21

2.4 Morphological TranSfOTTatIOI - - - - + + 31313 8+3 E*£#EE+vEEEeeEeeereesereeeeeeeerrrke 22

4.5 L298N Motor Driver: oc 25

CHAPTER Ê5 5-5 s9 GỌI 0n 00 000000 9090098090090 27

RESULT AND DISCUSSION 5< << HH m0 27

5.1 Problems Solved óc ng TH ng HH Ti ng Hư 28

5.2 Problems 09.0 Ả 28

CHAPTER Á 5-5 << << 9 9 4 000 909098090896 28

CONCLUSION 6c 5c cọc 0n 0000090090096 28

LIST OF FIGURES

Figure 1 Inverted pendulum cecccccccsessscsseeeseesseeeeecseecesecssceseaeceeeeesecsseeceseeeeeeensenaes 8

Figure 2.PID contrO ÏÏ€T- - - c2 2c 3221211351131 1 9111912111 11111 1 T11 T1 T1 HT HT ng 9

Figure 3.Models two wheels self balancing robot - - c + * 3 +vSesereeerseserresre 9

Figure 4.Analyze the forces acting on the two wheeÌS - -sS-ss+ssereeerreresersee 10

Figure 5.Pole-Zero Map 0 000888 16

Figure 6.Image processing techniques flowchatt :cceccesceseeseeeceeeseeseeeeeseeeeeseeeeeeseens 20

Figure 7 HSV Trackbar 1n 'ÃỶ 21

Figure 8.Block Diagram of Proposed System - «+ tr iey 23

Figure 9.Structure of Raspberry PP1 - + - 1x31 v 1 111911 91 1 11H ng ng ng nếp 24

Figure lo v.ìii00 (060A 5 25

Figure 11.L298N Motor Driver G2 1139118391189 111 911 1 11 1 1H ng ng vn rệp 26

Figure 12 Results 1117 Ò 27

DESIGN OF TWO-WHEELED SELF-BALANCING ROBOT

CHAPTER 1 INTRODUCTION

1.1 Problem Statement

Automation is becoming an indispensable development trend of the modern world,

gradually replacing human labor with automated machinery in production lines In this

context, the research and development of robots, especially mobile robots, have become asignificant focus in the automation industry One of the most interesting topics is the self-balancing two-wheeled vehicle—a design that not only presents technical challenges but

also has wide-ranging potential applications The self-balancing two-wheeled vehicle

needs to be designed with a controller that can maintain balance and stability on all terrains,

under all weather conditions, and in various situations This requires a deep understanding

of mechanics, electronics, and control algorithms, particularly the PID control algorithm,

to process signals from sensors and respond swiftly from the control system The success

of the self-balancing two-wheeled vehicle can help reduce accidents due to loss of control,increase efficiency in transporting goods in factories, and expand applications in everyday

life Recognizing the importance and challenges of researching self-balancing two-wheeledvehicles, we have chosen the topic "Design of a Self-Balancing Two-Wheeled Vehicle with

a PID Controller" as the focal point for our scientific research project Although we still

have limitations in knowledge and experience, we hope to receive understanding and

support from our teachers and peers.

1.2 Research Objectives

Gain a basic understanding of the concepts related to two-wheeled balancing robots

Apply knowledge from various courses to successfully design a controller for a wheeled balancing robot.

two-Understand the design process of a system and simulate it using Matlab

1.3 Research Methods

Review literature and online articles about self-balancing two-wheeled robots

Study the PID control algorithm.

Validate theories through simulation and proceed with the actual product development.

1.4 Structure of the Thesis

Chapter 1: Introduction to the Topic

Chapter 2: Theoretical Basis of PID Controllers

The inverted pendulum is a system where an object is allowed to rotate around an axis A

distinctive feature is that the center of gravity of this object is located above the pivot point,

creating an inherently unstable system Without precise control, the pendulum will freely

fall to one side under the influence of gravity

We have a diagram that describes the inverted pendulum system as follows:

Figure 1 Inverted pendulum

2.2 PID Controller

An effective system is one that operates optimally based on a specific standard, achieving

a desired maximum or minimum value Achieving and maintaining this optimal state

depends on quality requirements, external influences, and the operating conditions of the

system One of the control tools that helps establish optimal operation for the system is the

PID controller

To design a PID controller, a negative feedback system must be established, where the error value between the actual output and the desired setpoint is measured This error is input

into the PID controller, which then calculates and provides the necessary voltage to adjustthe motors, aiming to bring the system back to the desired equilibrium position

2.3 Models and Symbols

We have the following model of a two-wheeled self-balancing vehicle:

Figure 3.Models two wheels self balancing robot

Here are the symbols for the two-wheeled self-balancing robot:

Kí Hiệu Đơn vị Ý nghĩa

M, Kg Robot mass

My Kg Wheel mass

m Wheel radius

8, rad Tilt angle of the robot body

Ow rad Wheel rotation angle

Ow rad/s Angular velocity of the wheel

r Ohms Motor resistance

L H Motor inductor

Ky Nms/rad_ | Friction coefficient

Moment coefficient

K, Vs/rad Back emf coefficient of the motor

a rad/s? Angular acceleration of the engine

iA V Voltage supplied to the motor

V, V Hardware voltage

i A Hardware current

L, Kgm? Moment manages the engine

Tm Nm Moment of motor

lw Moment of inertia of the wheel

I, Moment of inertia of the vehicle body

H,,Hp, Py, Pr Force acting between the wheel and the robot

chassis

i Distance from the center of the wheel to the

center of gravity of the vehicle body

C,, Cp Motor torque acting on the wheel Ary, Apr Friction force between the wheel and the

surfaceCorrection coefficient

2.4 Dynamic model of DC motor

Apply Newton's laws to calculate the total force acting on the wheel in the horizontal

From the kinematic equation and the differential equation of the DC motor, we obtain the

output torque of the motor

Since linear motion exerts force on the motor shaft, angular velocity can be converted into

linear velocity according to the following equation:

From the two equations above, we can derive the equations for the left and right wheels as

Analysis of Forces Acting on the Wheel Body

Apply Newton's law to calculate the total force acting on the wheel in the horizontal

Perpendicular force acting on the vehicle body:

» Fey = M,XcosØ;,

(H, + Hạ)cos8, + (P, + Pạ)sin8, — Mygsin@, — M,16, = M,3cosØ, (14)

Total torque acting on the center of gravity of the vehicle body:

Substituting into equation 15, we have:

—(H, + Hp)lcos6, — (P, + Pg)lsin8, — ( —tkme + 2m _ :

Equations (18) and (19) represent the nonlinear system of equations for the system To

linearize the model above, we 6, = 7 + where 9 is a small angle oriented vertically

upward From this, we have:

“ny, = (2M, + Sở + My) # +=

Rr2 # — My ld (21)

To derive the state-space model of the system, we extract the state variables from equations

(20) and (21), resulting in:

2ly

B = (2M, + +M,)

ly

a= |I,8 + 2M,lf (M, + 3]

With the model above, it is only accurate when the vehicle remains on a flat surface,

disregards the friction between the wheels and the surface, and when the external forces

are negligible

CHAPTER 3

3.1 Declaration, System Analysis, and Controller Design

To build a controller, the first thing we need to do is declare the system values and assess

whether the system is controllable, observable, and stable

14

18 alpha = Ip*beta+2*Mp*1*2* (Mw+Iw/r’2) ;

3.1.2 System Stability Analysis

Transfer Function HTXE1 =

-8.0233

15

We can see the roots of the transfer function and its poles, from which we can conclude

that according to the Routh-Hurwitz criterion, the system is unstable.

-25 -20 -15 -10 -5 0 5 10 15 20

Real Axis (seconds !)

Figure 5.Pole-Zero Map

3.1.3 Analysis of System Controllability Based on the principles of automatic control

for a linear system with a state-space equation as follows:

x =Ax+ Bu

y=Cx+Du

With A € RTM, Be RTM, Ce RTM,DeERTM

Construct the controllability matrix:

P = [B, AB, A?B, , A" 1B]

The necessary and sufficient condition for a system described by state-space equations to

be controllable is that the rank of the controllability matrix P equals, where n is the number

of state variables in the system

We use MATLAB to verify the controllability of the system.

16

5

ole gle

% -Khao sat tinh dieu khien duoc cua he

thong -P = [B A*B A*2*B A*3*B]

The rank of matrix P equals the number of state variables of the system

Analysis of System Observability

Construct the observability matrix:

L = [C,AC, A?C, , A"~1C]

We use MATLAB to verify the observability of the system

% -Khao sat tinh quan sat duoc cua he thong -%

L = [C; C*A; C*A^2; C*A*3];

rank (L)

17

The rank of the matrix P is equal to the number of state variables in the system.

Comments: From the two verifications, we can conclude that the system is controllable and

all four state variables of the system can be observed By designing the matrix C to observetwo state variables—the position of the vehicle and the tilt angle of the vehicle—if thesetwo variables are stable, then the entire system will be stable This approach ensures that

by monitoring these critical variables, we can gauge the overall system stability.

18

BALL DETECTION AND TRACKING USING RASPBERRY PI

CHAPTER 1 INTRODUCTION

New technology is developing quickly as the future gets closer the development of new

age robotics, artificial intelligence, and the internet of things As is customary, robots are

crucial to industry since they are assigned certain tasks Robots are employed extensively

in many industries these days, including aerospace, defense, medical, and space travel.Current robots are costly due to their sophisticated mechanisms and specialized nature Our

suggested work is reasonably priced and comes with standard sensors like a camera, servomotor, and DC motor, among others Numerous applications, including video surveillance,human-computer interaction, vehicle navigation, and robot control, rely heavily on ball

tracking The technology is designed to accurately identify and track the item The item is

detected by the webcam Information is transmitted to Raspberry pi 3 after detection The

item is tracked with the aid of a DC motor It is employed to track several items with various

sizes, colors, and structural configurations But in real-world applications, a variety ofelements, including changes in lighting, appearance, shape, partial occlusion, and cameramotion, complicate the issue

A robot with legs is more flexible than one with four legs It can maneuver in manydirections and across a variety of terrains, including confined areas These days, a lot of

companies are replacing human labor with this kind of robot to carry out jobs like carrying

items and doing activities in dangerous areas As a result, there are many possible

applications Numerous applications in fields like sports, health, etc., that facilitate

communication between humans and computers, assess patient motions during

recuperation, operate robots, and guide automobiles mostly rely on ball tracking The

purpose of this technology is to precisely identify and track objects

19

CHAPTER 2 RESEARCH METHODOLOGY

Read image from

Vebcam

Filtering out color of the desired object

Find the center

Identify object and

draw circle

Desired object

Figure 6.Image processing techniques flowchart

The procedures for software setting are:

2.1 Receiving Image From Camera

The raspberry will use the aforementioned software script to collect picture data from its

internal camera and save it in the camera variable If the camera does not provide any data,

it will take a break

if not args.get("video", False):

vs = VideoStream(src=0).start() else:

vs = cv2 VideoCapture(args["video"])

while True:|

frame = ys.read() frame = frame[1] if args.get("video", False) else frame

if frame is None:

break

The HSV color picker program is used to find the ball color's HSV value An illustration

of how to use the color picker program to select colors is as follows:

2.3 Thresholding

The method of thresholding transforms a hsv color picture into a binary image in which

the detected color image turns white if it matches a preset range value and black if it doesnot The software script that was utilized in this investigation is as follows:

mask = cv2.inRange(hsv, greenLower, greenUpper)

The previously defined hsv color space is utilized as the range value The masking process

is another name for the thresholding procedure

21

2.4 Morphological Transformation

Is a technique that increases the objectification of the masking by using the erosion and

dilation functions Little color spots will be eliminated by the erosion mechanism, leaving

a vibrant color cast The collection of colors that the erosion function has pre-sorted is then expanded by combining this with the dilation function In order to apply it to ball detection,

the detection will be more ball-focused The study employed the subsequent software script,

which is as follows:

mask = cv2.erode(mask, None, iterations=2)

mask = cv2.dilate(mask, None, iterations=2)

2.5 Draw Contour

Lines that connect all locations with the same intensity along the image's border are

referred to as contours Shape analysis, object detection, and determining the size of things

of interest can all benefit from contours Once every stage of the prior image processinghas been mastered, contours may be produced The following is the script for the draw

contours software used in this study:

ents = cv2.findConfours(mask.copy() cv2.RETR_ EXTERNAL,

cv2.CHAIN APPROX SIMPLE)

2.6 Find the center

The following program script used in this study is as follows:

center = (int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"]))

2.7 Draw Circle

The following step involves creating a circle with the parameters of the previously discovered contour findings The cv2.circle function is used to do this, and the

cv2.minEnclosingCircle function is used to determine the circle's radius and center point

position The study employed the subsequent software script, which is as follows:

cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 255), 2) ((x, y), radius) = cv2.minEnclosingCircle(c)

22